Catalini Models Measurability Gap in AI Economy

According to a working paper titled "Some Simple Economics of AGI" by Christian Catalini of MIT Sloan School of Management, the paper models a structural mismatch between the rapidly falling cost of agentic AI automation and the relatively flat, biology-constrained cost of human verification. The paper names that mismatch the Measurability Gap and identifies a mechanism Catalini calls the "Trojan Horse" externality, where firms optimising for visible metrics deploy unverified AI outputs that accumulate low-quality data and produce a so-called "hollow economy." The paper contrasts that scenario with an alternative "augmented economy" driven by human oversight.

Editorial analysis: The paper frames the economic transition as a race between two cost curves, one for automation that declines steeply and one for human verification that plateaus. This modelling choice highlights measurability and auditing as core frictions rather than just task-replacement. The working paper uses that formalism to show how incentives that reward speed, scale, or throughput can produce feedback loops of unverified outputs; the GlobeNewswire and Manila Times summaries reproduce that mechanism and Catalini's terminology.

The Measurability Gap argument reframes several ongoing debates - from dataset provenance and model evaluation to observability and ML Ops - by putting verification costs at the center of economic incentives for deployment. For practitioners, this suggests quality-control externalities extend beyond immediate model performance to ecosystem-level data degradation, particularly where automated agents create training or operational data at scale.

For practitioners and policymakers: indicators to monitor include prevalence of automated data-generation pipelines without independent verification, metrics used in procurement that prioritize throughput over auditability, and the emergence of institutional checks such as independent validation services, verifiable provenance metadata, or regulation that mandates sampling and human-in-loop verification. Catalini's paper sets conceptual markers rather than prescriptive firm-level steps; the author's direct quote captures the shift in human roles: "When getting an answer to a question costs almost nothing, the value of human work shifts to knowing what to ask and being certain the result is correct," Catalini said.

Editorial analysis: Broadly, the paper supplies a compact economic vocabulary for conversations about RLHF, agentic systems, and governance. Observers designing production ML systems should watch whether verification costs become binding constraints as deployments scale, and whether market incentives reward verifiability as opposed to raw throughput.