GPT-4 Spear-Phishing Texts Matched Novice Human Writers in Pilot
A peer-reviewed pilot published on August 1 found that 25 participants were no better than chance at identifying whether personalized spear-phishing texts came from GPT-4 or novice human writers. Estimated click intent was 28% for AI-generated messages and 21.3% for human-written ones, but that difference was not statistically significant; job-themed messages were the clearest risk signal.
A peer-reviewed pilot study published on August 1 found that people could not reliably distinguish personalized GPT-4 spear-phishing texts from messages written by novice humans. The result does not show that GPT-4 was definitively more persuasive, but it suggests that a simple automated prompt can produce targeted messages in the same performance range as time-limited student writers.
The study, published in the Journal of Cybersecurity and Privacy, used a method the authors call the Threshold Ranking Approach for Personalized Deception. Twenty-five consenting participants provided details about their work, hobbies and recent social-media activity. Researchers used those details to create personalized SMS phishing messages. Each participant then ranked six GPT-4 messages and six messages written by undergraduate cybersecurity students.
What the numbers show
The model estimated that participants would click 28% of GPT-4 messages and 21.3% of the human-written messages. The difference was not statistically significant, so the pilot cannot establish that GPT-4 outperformed the human group. The comparison was also against novice student writers working under time constraints, not professional social engineers.
Message topic mattered more clearly. Job-related messages reached an estimated 38% click-intent rate, compared with 19% for hobby messages and 17% for messages based on social-media posts. Participants identified message authorship correctly 52% of the time, essentially the 50% expected from guessing.
The researchers also found that the two message sets were computationally distinguishable within this controlled dataset. Help Net Security reported 88.7% balanced accuracy for a study-specific classifier under the strictest preprocessing and participant-separated test setup. That is not evidence for a general-purpose AI-text detector: the classifier was trained on one model, one prompt design and one pool of human writers.
Limits for defenders
No live malicious links were sent. Participants sorted printed messages and stated whether they would click, so the study measured intention rather than observed compromise. The sample was small, and the exact GPT-4 snapshot and generation logs were not preserved.
The practical signal is narrower than a headline claiming AI won. Personalized phishing can be automated cheaply enough to reach human-like persuasiveness in a small controlled test, while recipients' intuition about authorship offered no useful protection. Defenses should verify the sender, channel, link and requested action instead of relying on whether a message sounds machine-written.
Key Points
- 1In a 25-participant pilot, estimated click intent was 28% for GPT-4 messages and 21.3% for novice human-written messages, but the difference was not statistically significant.
- 2Job-themed messages produced a 38% estimated click-intent rate, above hobby and social-media themes, while participants identified AI authorship only 52% of the time.
- 3The experiment used printed messages and stated intent, not live attacks, and its study-specific text classifier does not establish a general AI-detection method.
Scoring Rationale
The peer-reviewed pilot offers a useful controlled signal about low-cost personalized phishing and human detection limits, while its small sample, novice comparison group and stated-intent design limit generalization.
Sources
Primary source and supporting public references used for this report.
Practice interview problems based on real data
1,625 SQL & Python problems across 15 industry datasets — the exact type of data you work with.
Try 250 free problems
