Executives Measure AI ROI Using Outcomes Not Tokens

Business Insider spoke with four executives about how they measure AI ROI at Mistral's AI Summit in Paris, according to the published report. Charles Holive, chief AI officer at BNP Paribas CIB, told Business Insider, "We try to go away from vanity metrics - billions of tokens per day." Holive added, "What did you do, you didn't do before? How much faster did you do it?" Antoine Pichot, director of innovation, digital and data at La Banque Postale, told Business Insider the bank measures whether AI makes employees more efficient, improves customer service, and delivers value for money. Amit Kapur, chief AI and transformation officer at Tata Consultancy Services, told Business Insider he focuses on whether AI improves business performance rather than token consumption. Sujay Bhattacharya, executive managing director at NTT DATA and a leader working with Mistral AI, told Business Insider that his customers are increasingly looking beyond token counts and focusing on total cost and business value.

Companies and enterprise teams have commonly tracked low-level usage metrics such as API calls or token volumes as an early proxy for adoption. Industry reporting and the executive comments in Business Insider reflect a shift in public conversation toward outcome-oriented KPIs, where measures tied to productivity, customer outcomes, and cost-per-impact matter more than raw model consumption.

For practitioners, moving the conversation from token counts to outcome metrics typically entails instrumenting feature-level telemetry and tying model outputs into business KPIs. Observed patterns in similar transitions include building measurement layers that capture latency, error rates, time saved, automation rates, and downstream business metrics such as conversion lift or cost-per-ticket. These are implementation challenges familiar to data and ML teams integrating LLMs into workflows.

• •Evidence of standardised outcome KPIs for AI within enterprises, such as time-to-decision, resolution time, or revenue-per-employee.
• •How engineering and ML teams map model-level metrics (latency, token usage) to business outcomes in dashboards and SLAs.
• •Vendor and procurement responses: whether contracts, pricing, and tooling shift from token-based models to outcome- or value-based pricing.

The executives quoted in Business Insider illustrate a pragmatic framing: reporting and instrumentation that tie model outputs to concrete business outcomes make ROI conversations actionable. Teams planning evaluation should prioritise defining measurable business outcomes and the telemetry needed to attribute changes to AI interventions.