Why it matters: The U.S. Special Operations Command’s pursuit of AI-generated deepfake internet users for information gathering raises serious concerns about privacy, propaganda, and the erosion of public trust. This technology could potentially be used to spy on American citizens and spread misinformation, contradicting the government’s own warnings about the dangers of deepfakes.
The plan unveiled: A recently revealed procurement document shows that the Pentagon is seeking technology to create convincing online personas indistinguishable from real people. These AI-generated profiles would be designed to:
- Bypass social media verification checks
- Include multiple expressions and government ID-quality photos
- Generate selfie videos to prove “liveness”
Intended use and potential misuse: While The Intercept reports that the stated purpose is to gather information from public online forums, experts warn of broader implications:
- Potential for domestic surveillance: The technology could be used to infiltrate online communities and monitor American citizens. (Americanmilitarynews)
- Spread of misinformation: AI-generated profiles could be used to disseminate propaganda or influence public opinion.
- Undermining election integrity: Deepfake technology poses a significant threat to fair and transparent elections.
Hypocrisy and eroding trust: The Pentagon’s pursuit of deepfake technology directly contradicts previous government warnings about its dangers:
- U.S. intelligence agencies have labeled deepfakes a “top risk” for modern communications.
- The government has condemned the state-backed use of deepfakes by other countries.
- This apparent double standard could further erode public trust in government communications.
Daniel Byman, a member of the State Department’s International Security Advisory Board: “There is a legitimate concern that the U.S. will be seen as hypocritical.” Byman said, “I’m also concerned about the impact on domestic trust in government — will segments of the U.S. people, in general, become more suspicious of information from the government?”
Privacy concerns: The creation of convincing deepfakes requires vast amounts of biometric data:
- Hundreds to thousands of images are needed to generate a believable fake persona.
- This raises questions about how the government might collect and use citizens’ personal data.
Defending democracy: Civil liberties advocates argue that the use of deepfakes by the military poses a significant threat to democratic principles:
- It blurs the line between truth and fiction in public discourse.
- It could be used to manipulate public opinion or discredit legitimate information.
- The technology’s potential for abuse outweighs any perceived benefits for national security.
The bottom line: While the Pentagon argues that this technology is necessary for national security, its development and potential use raise serious ethical concerns. The American public must remain vigilant and demand transparency about how such powerful tools might be employed, especially when they have the potential to infringe on civil liberties and undermine the foundations of democratic society.
