Databricks Releases OfficeQA Pro V2 for Enterprise Reasoning
Databricks released OfficeQA Pro V2, a 90-question benchmark built on 1,435 U.S. federal receipts and expenditures documents spanning 1793 to 2024. Most questions require evidence from multiple documents, and Databricks reports that its best tested agent harness reaches 54.4% accuracy, leaving substantial room for improvement in retrieval, parsing and calculation.
Databricks has released OfficeQA Pro V2, a grounded-reasoning benchmark designed to test whether AI agents can retrieve and combine evidence across difficult financial documents. The new version contains 90 questions and uses a different corpus from the original OfficeQA Pro: 1,435 U.S. federal receipts and expenditures documents covering 1793 through 2024.
A harder generalization test
The corpus combines historical statements of government receipts, outlays and balances with earlier congressional serial-set records. According to the dataset card, its 90 questions reference a median of 5.5 source documents, with individual questions requiring between one and 24 documents. The source material includes dense tables, revisions across long time spans and older scans that can be difficult to parse reliably.
That design makes V2 more than a larger question set. Because it uses a separate corpus from the Treasury Bulletins behind the original benchmark, it can test whether an agent has learned a transferable retrieval-and-reasoning workflow instead of overfitting its tools to one document collection. Databricks says the best agent harness in its current results reaches 54.4% accuracy. That is a company-reported benchmark result, not an independent evaluation.
What teams can evaluate
The dataset is hosted on Hugging Face under a gated-access agreement intended to keep answer keys away from agents being evaluated. Databricks provides the original PDFs, parsed JSON representations and a scoring function in its public GitHub repository. The dataset card says the full PDF collection is about 13.3 GB, while the parsed JSON corpus is about 794 MB.
For practitioners, the useful signal is the benchmark's separation of retrieval, document parsing and numerical reasoning failures. A system can find a plausible figure and still fail if it overlooks a later revision, misreads a table boundary or combines values from the wrong years. Teams using the benchmark should therefore report the model, agent harness, document representation, retrieval tools and scoring tolerance alongside any headline accuracy. Without those details, scores from different implementations are not directly comparable.
Key Points
- 1OfficeQA Pro V2 contains 90 questions grounded in 1,435 U.S. federal receipts and expenditures documents spanning 1793 to 2024.
- 2Questions reference a median of 5.5 source documents, with a range of one to 24, to stress retrieval and multi-document reasoning.
- 3Databricks reports 54.4% accuracy for its best current agent harness; the result is vendor-reported and depends on the disclosed evaluation setup.
Scoring Rationale
The release provides a reproducible, licensed benchmark for multi-document enterprise reasoning over a substantially different corpus, with direct value for evaluating retrieval and parsing systems; impact is moderated because the current performance figures are vendor-reported.
Sources
Primary source and supporting public references used for this report.
Practice interview problems based on real data
1,625 SQL & Python problems across 15 industry datasets — the exact type of data you work with.
Try 250 free problems
