Llama-3 Improves Intent Detection For Smoking-Cessation
AI-assisted, source-derived brief produced by the Let's Data Science Automated News Desk. The source material used is linked on this page.
- Source event:
- first reported
- LDS brief:
- publication time is not available in the public LDS lifecycle record
In a 2026 study, UC Irvine researchers fine-tune Meta's Llama-3 8B to detect message intents in a smoking-cessation support-group dataset with severe class imbalance. Baseline F1-scores were low (unweighted 0.41, weighted 0.38 on downsampled data), while fine-tuning with downsampling and annotation correction raised F1 up to 0.88 unweighted and 0.91 weighted; full-dataset F1s remained lower (0.57–0.66). The work shows domain-specific fine-tuning and balancing improve chatbot intent recognition for mHealth interventions.
Key Points
- 1Fine-tunes Llama‑3 8B achieves up to 0.88 unweighted and 0.91 weighted F1 on balanced data
- 2Shows dataset imbalance substantially reduces full-dataset performance, dropping F1 to 0.57 unweighted
- 3Suggests downsampling plus targeted human annotation correction improves recall, though automated downsampling may suffice
Scoring Rationale
Practical fine-tuning and downsampling yield strong results; impact limited by single-domain focus and persistent class imbalance.
Sources
Public references used for this report.
Practice interview problems based on real data
1,625 SQL & Python problems across 15 industry datasets — the exact type of data you work with.
Try 250 free problems