AI Forensics Finds Hugging Face Spaces Can Generate Nonconsensual Sexualized Images
AI Forensics reported on July 28 that seven of nine image-editing Spaces it tested on Hugging Face produced a topless edit after a simple request. Its separate seven-day honeypot logged more than 1,000 submissions, 73% of them sexual. Hugging Face called the findings a developer-safeguard gap but disputed parts of the methodology and said the sample was not representative of platform-wide use.
AI Forensics reported on July 28 that seven of nine image-editing Spaces it tested on Hugging Face produced a topless edit of a clothed photo after a simple request. WIRED reported that the researchers used the same six-word prompt in each test and did not try to bypass safeguards.
The finding concerns public Hugging Face Spaces, which let developers host models behind usable web interfaces. The tested tools were presented as general image editors rather than services explicitly marketed for creating nonconsensual intimate images.
A honeypot measured attempted misuse
AI Forensics also created image-editing Spaces that logged submissions but did not generate images. Over seven days, the nonprofit collected more than 1,000 prompts and uploaded images. According to the investigation and WIRED's reporting:
- •73% of the prompts were sexual;
- •83% of the sexual requests sought to undress or sexualize an uploaded person;
- •95% of those requests targeted women; and
- •6.7% of the sexual requests targeted apparent children.
These figures describe requests sent to the research honeypots. They are not an estimate of the share of all Hugging Face users or Spaces engaged in abuse, and the classification involving children was based on apparent age.
Hugging Face disputes the scope of the findings
WIRED updated its report after publication with Hugging Face's response. The company said the investigation showed a gap in safeguard adoption by some image-editing developers and that it was working to address that gap. It also questioned the researchers' methodology, said the collected prompts were not representative of platform-wide use and argued that scanning every prompt and output centrally would not be technically feasible across the varied code running in Spaces.
Hugging Face's published content policy prohibits sexual content created without explicit consent and any sexual content involving minors. WIRED reported that some pages promoting nudifying services were removed after it contacted the company, but it could not establish whether those removals were connected to the inquiry.
The deployment layer is part of the safety boundary
The evidence supports a deployment-risk conclusion, not a platform-wide prevalence estimate. Public model interfaces need controls at several layers: request filtering, output review, abuse logging, rate limits, reporting channels and rapid takedown procedures. The disagreement over whether those controls should sit with each developer or the hosting platform is unresolved, but the observed tests show why teams should evaluate the deployed application rather than relying only on a base model's documentation.
Key Points
- 1Seven of nine Hugging Face image-editing Spaces tested by AI Forensics produced a topless edit after a simple request without an attempted safeguard bypass.
- 2The nonprofit's seven-day honeypot logged more than 1,000 submissions; 73% were sexual, but the sample is not a platform-wide prevalence estimate.
- 3Hugging Face acknowledged a safeguard-adoption gap among some developers while disputing the methodology and representativeness of the findings.
Scoring Rationale
The originating investigation documents serious misuse risk in hosted image-editing applications and measures both tool behavior and submitted abuse attempts. The platform response and bounded sample materially limit broader prevalence claims, while the deployment-control implications remain important for teams operating public image tools.
Sources
Primary source and supporting public references used for this report.
Practice interview problems based on real data
1,625 SQL & Python problems across 15 industry datasets — the exact type of data you work with.
Try 250 free problems
