Revolut Deploys PRAGMA Foundation Model for Finance

Revolut published PRAGMA, a family of Transformer-based foundation models tailored to multi-source banking event sequences, and an arXiv preprint describes the approach, training objective, and downstream evaluation. The model is pre-trained with a masked modelling, self-supervised objective designed for the discrete, variable-length nature of transaction and event logs, and is reported to be trained on a large corpus of roughly 40 billion behavioral events. The paper shows PRAGMA yields strong embeddings that enable good downstream results via linear probes and improve further with lightweight fine-tuning.

The pre-training uses a Transformer backbone adapted to sequence heterogeneity and sparse, event-driven timestamps. The self-supervised objective is a masked modelling variant tuned for variable-length, categorical-rich records rather than continuous text tokens. Key practitioner takeaways:

• •Models are trained on multi-source banking event sequences to capture cross-product signals (payments, authorization events, support interactions).
• •Embeddings produced by PRAGMA support downstream tasks including credit scoring, fraud detection, and lifetime value prediction with minimal additional training.
• •Simple linear models on top of frozen embeddings already achieve strong baselines; lightweight fine-tuning yields consistent uplift.

Evaluation and infra: The preprint reports broad evaluation across multiple financial domains and shows superior performance compared to task-specific baselines. Revolut also describes a production inference stack capable of high-throughput, low-latency use: industry reporting indicates inference and orchestration run on an AI cloud with more than 200 NVIDIA H100 GPUs, plus tooling for monitoring and FinCrime agents that leverage shared embeddings.

Vertical, domain-specialized foundation models are an accelerating trend. PRAGMA is a clear example of a financial vertical model where proprietary behavioral signals produce a material performance moat. By focusing on raw event sequences and representation learning, Revolut moves evaluation emphasis from retrieval augmentation and large general LLMs toward robust, reusable embeddings that scale across supervised tasks. This aligns with industry shifts where orchestration, observability, and domain data quality matter more in production than marginal improvements in open-domain generative quality. The result is a strategic asset: shared embeddings reduce duplicated effort across risk, product, and support teams and convert raw event scale into an operational advantage.

Risks and governance: Specialized models trained on transactional data raise model-risk, privacy, and regulatory scrutiny. Data lineage, explainability, drift detection, and auditability will be essential before PRAGMA-powered decisions drive credit or denials at scale. Practitioners will need to integrate model risk frameworks, differential privacy or synthetic data strategies, and rigorous evaluation across demographic slices.

Whether Revolut releases code, model checkpoints, or evaluation suites; how competitors respond with their own event-sequence models; and how regulators treat productionized, proprietary financial foundation models. Also monitor reproducibility and benchmark comparisons on public or synthetic banking datasets.

"careful data fetching and context engineering often matter more in production than heavy RAG patterns," said Pavel Nesterov, Executive Director of AI at Revolut, summarizing the trade-offs teams face when moving from experimentation to production.