ETH Zurich Trains Open LLMs on Alps
Researchers from ETH Zürich and EPFL this week revealed they trained two open large language models using Switzerland's Alps supercomputer at the International Open-Source LLM Builders Summit in Geneva. The models — roughly 8 billion and 70 billion parameters trained on about 15 trillion tokens across 1,000+ languages (40% non-English) — exploited Alps' Nvidia GH200 Superchip architecture and FP8 sparse performance. The team plans a fully open release this summer under Apache 2.0, including training code and transparent data.
Key Points
- 1Train two open LLMs (8B and 70B) on Alps supercomputer using 15 trillion tokens
- 2Leverage Nvidia GH200 Superchips and FP8 sparse compute for high-performance large-scale training
- 3Release models, training code, and transparent data under Apache 2.0 to enable reproducibility
Scoring Rationale
High-impact open-model release using a national supercomputer; originality tempered by prior open LLMs and similar infrastructure efforts.
Sources
Public references used for this report.
Practice with real Logistics & Shipping data
90 SQL & Python problems · 15 industry datasets
250 free problems · No credit card
See all Logistics & Shipping problems