Talkie Releases 13B Vintage Language Model Trained on 1930 Data

According to the project's GitHub repository and the HuggingFace model card, the talkie project releases talkie-1930-13b-base, an open-weight 13 billion-parameter language model trained exclusively on English-language books, newspapers, journals, patents, and case law published before December 31, 1930 (GitHub; HuggingFace). The HuggingFace model card reports the base model was trained on 260 billion tokens and is released under an Apache-2.0 license (HuggingFace). The project also publishes an instruction-tuned checkpoint, talkie-1930-13b-it, and a modern-web comparator talkie-web-13b-base, per the repository README (GitHub). The Register quotes the Talkie team as writing, "Talkie is the largest vintage language model we are aware of, and we plan to continue scaling significantly," and a public demo is hosted at talkie-lm.com (The Register; MarkTechPost).

Per the GitHub documentation and HuggingFace model cards, the instruction-tuned checkpoint talkie-1930-13b-it was produced from instruction-response pairs extracted from pre-1931 reference works and reportedly underwent reinforcement learning using online DPO with an LLM-as-a-judge during post-training (GitHub; HuggingFace). The README lists runtime requirements including Python >= 3.11, PyTorch >= 2.1, and GPUs with roughly 28 GB VRAM for bfloat16 inference; Simon Willison notes the talkie-1930-13b-it checkpoint is about 26.6 GB compressed for download (GitHub; Simon Willison weblog).

Editorial analysis: Vintage LLM projects create a controlled experimental setting for testing temporal generalization, dataset contamination, and how models hallucinate or extrapolate from historically bounded corpora. Observers writing about talkie highlight two recurring research themes: isolating training data by cutoff date reduces modern-web contamination when evaluating generalization, and deliberately archaic corpora reveal how training-distribution priors shape world models and factual inference (DTNS; startupfortune; The Decoder).

Reported testing coverage summarized by multiple outlets finds talkie often replies in a pre-1931 worldview, for example downplaying the likelihood of a second world war and imagining 2026 with steamships and extensive rail networks (The Decoder). Early evaluations reported by byteiota and The Decoder indicate the model can produce simple Python and cipher solutions but underperforms contemporary benchmarks and shows evidence of some post-1930 data contamination (byteiota; The Decoder).

For practitioners: track the GitHub and HuggingFace repositories for updated model cards and reproducibility assets (GitHub; HuggingFace). Observers will want to verify the claimed 260 billion token training corpus with dataset manifests and sampling statistics, inspect the instruction-tuning recipe and DPO logs for reproducibility, and monitor community evaluations for benchmark performance and contamination audits. Industry watchers may also follow the teams stated scaling plans and any larger checkpoints or datasets that the project releases, as noted in public commentary (The Register).

Talkie is a deliberately constrained, open-weight 13B model trained on pre-1931 English-language text and released with documentation and checkpoints on GitHub and HuggingFace. The project supplies a practical testbed for research into temporal generalization and dataset contamination, and early community tests show both interesting historical behavior and limitations versus modern models.