Classification of Threats from Deepfakes

Synthetic media threats such as deepfakes pose a growing challenge to all users of modern technologies and communications. As with many technologies, synthetic media techniques can be used for both beneficial and malicious purposes -- a reality highlighted in the paper "Contextualizing Deepfake Threats to Organizations" published by the NSA and FBI, which is summarized in this article. While there is limited evidence of significant use of synthetic media by state-sponsored actors, the increasing availability and effectiveness of these techniques suggest that less sophisticated cyber actors will adopt them with growing frequency and refinement. Synthetic media threats primarily involve technologies related to text, video, audio, and images used online and across all types of communication channels. Deepfakes are a particularly concerning category of synthetic media that leverage artificial intelligence and machine learning (AI/ML) to create believable, highly realistic content. The most serious threats from synthetic media misuse include techniques that compromise an organization's brand, impersonate executives and financial officers, and use fraudulent communications to gain access to networks and sensitive information.

Organizations can take several steps to identify, defend against, and respond to deepfake threats. They should consider deploying a range of technologies to detect deepfakes and determine the provenance of media, including real-time verification capabilities, passive detection techniques, and active authentication methods. Organizations can also minimize the impact of malicious deepfake techniques through information sharing, planning and rehearsing responses to exploitation attempts, and targeted employee training. Deepfake-enhanced phishing in particular will pose an even greater challenge in the future, making proactive preparation essential for identifying and combating these attacks.

Types of Deepfake Threats

Several terms are used to describe media that has been synthetically created or manipulated. Among the most common are: deepfakes, faux-to, and AI-generated media.

Superficial Fakes

Multimedia content manipulated using techniques that do not rely on machine learning or deep learning is often referred to as superficial or cheap fakes (shallow fakes). Despite their simplicity, these can be just as effective as more technically sophisticated methods. They are typically created by altering an original message within authentic media. Examples include:

• Selectively copying and pasting content from an original scene to remove an object from an image, thereby changing the narrative
• Slowing down a video by adding repeated frames to make a person appear intoxicated
• Combining audio clips from a different source and replacing the audio track in a video to alter the message
• Using fabricated text to promote a false narrative and cause financial or other harm

Deepfakes

Multimedia content that has been either fully synthesized or partially edited using machine learning and deep learning (artificial intelligence) is referred to as deepfakes. Notable examples include:

• LinkedIn saw a significant increase in deepfake profile pictures in 2022.
• An AI-generated image depicting an explosion near the Pentagon was shared online in May 2023, causing widespread confusion and brief stock market volatility. The image was a so-called hallucination -- fabricated content that appears plausible but is entirely fictitious.
• A deepfake video showed Ukrainian President Volodymyr Zelenskyy calling on his country to surrender to Russia.
• Several Russian television and radio stations were hacked, and an alleged deepfake video of President Vladimir Putin was broadcast claiming he had declared a state of emergency due to an invasion of Ukraine.
• Text-to-video diffusion models represent another advancing technology, enabling the creation of fully synthetic AI-generated videos.
• In 2019, deepfake audio recordings were used to steal $243,000 from a UK company. Since then, personalized fraud using sophisticated AI voice cloning models has surged dramatically.
• Openly available Large Language Models (LLMs) are now being used to generate text for phishing emails.

Generative AI

Since 2023, Generative AI has gained significant traction for its ability to produce synthetic media. Techniques such as Generative Adversarial Networks (GANs), Diffusion Models, and Large Language Models (LLMs) -- or combinations thereof -- enable the production of highly realistic synthetic multimedia content trained on vast datasets.

Computer Generated Imagery (CGI)

CGI refers to the use of computer graphics to create or enhance visual media (images and video). Traditionally, these methods have been the standard for visual effects in major motion pictures. However, as Generative AI techniques become more capable and affordable, the two technologies are converging to produce even more convincing fakes.

Detection vs. Authentication

For several years, public and private organizations have raised concerns about tampered multimedia content and developed means to detect and counter it. Numerous partnerships between public and private stakeholders have emerged, focusing on cooperative efforts to identify manipulations and verify or authenticate media content. Detection and authentication serve different goals, and the key distinction is straightforward: detection methods are typically passive forensic techniques, while authentication methods are active forensic techniques embedded at the time media is captured or edited. Detection efforts focus on developing methods that look for signs of tampering and present findings as numerical outputs or visualizations, alerting analysts that media requires further examination. These methods are built on the assumption that modifications to original data or fully synthetic media leave statistically significant traces. This type of detection is inherently a cat-and-mouse game: as new detection methods are published, countermeasures quickly follow. Until authentication standards are universally adopted, however, these detection methods remain essential for forensic analysis.

Authentication methods are designed to be embedded at the time of capture, creation, or editing, making the provenance of media transparent. Examples include digital watermarks in synthetically generated media, active signals in real-time recordings for liveness verification, and cryptographic asset hashing at the device level.

How Deepfakes Can Threaten Organizations

Public concern about synthetic media also extends to their use in disinformation operations designed to spread false information about political, social, military, or economic issues, causing confusion and uncertainty. However, the synthetic media threats that organizations most commonly face involve activities that directly jeopardize the brand, financial standing, security, or integrity of the organization itself. The most significant synthetic media risks -- particularly for the defense industry and critical infrastructure organizations -- include:

Executive Impersonation for Brand Manipulation

Malicious actors can use audio and video deepfakes to impersonate executives and other senior personnel. By creating convincing impersonations of key leaders and rapidly disseminating them via social media, attackers can damage an organization's reputation and brand value before the content can be stopped or debunked. Manipulated media operations targeting high-profile political figures such as Ukrainian President Volodymyr Zelenskyy have already demonstrated how effectively this technique spreads disinformation and confusion. The impact can be particularly severe for international brands whose stock prices and overall reputations are vulnerable to disinformation campaigns. Given the potential consequences, this type of deepfake is a significant concern for many CEOs and government leaders.

Manipulation for Financial Gain

Malicious actors -- many of them cybercriminals -- frequently use various forms of manipulated media in social engineering campaigns for financial gain. This includes impersonating key executives or financial officers using manipulated audio, video, or text to authorize the fraudulent release of funds to attacker-controlled accounts. Business Email Compromise (BEC) is a common variant of this social engineering technique and has cost organizations hundreds of millions of dollars in losses. Similar methods can also be used to manipulate cryptocurrency trading. In practice, these types of scams are widespread, with multiple partners reporting that they have been targeted by such operations.

Manipulation for Unauthorized Access

Malicious actors can apply the same manipulated media techniques to gain access to an organization's employees, operations, and information. This includes the use of synthetic media during job interviews, particularly for remote positions. In 2022, attackers reportedly used synthesized audio and video in online interviews, though the content was often incoherent or poorly synchronized -- revealing the fraudulent nature of the calls. These attempts were enabled by previously stolen personal information.