EnterpriseVal measures generative AI fit and value for businesses
EnterpriseVal: Quantifying the Efficacy, Reliability and Value of Generative AI in the Enterprise
Artificial Intelligence
Summary
Many businesses use generative AI but struggle to know if it really helps or if it's safe and reliable for their specific tasks. The authors designed EnterpriseVal, a system that tests AI tools in real company workflows by considering the exact setup, risks, and quality needed. It combines expert reviews and AI scoring to decide whether AI tools are good enough to use widely or need improvement. They tried it in a global bank and saw improved accuracy and big time savings for some tasks. Their work helps companies make better decisions about using AI effectively and safely.
What this means in practice
- •For enterprise ai teams: Evaluate generative AI tools realistically in real business workflows to decide if they should be adopted, scaled, or rejected.
- •For banking operations teams: Reduce manual effort and improve accuracy in credit memo creation by assessing AI-generated drafts with measured precision and hallucination rates.
Authors
Abbas Raza Ali, Muhammad Ajmal Siddiqui, Moona Zahid
Abstract
Frontier language models now produce professional deliverables that expert graders judge to match human work on a substantial share of economically valuable tasks, yet most enterprise GenAI initiatives fail to show a measurable business effect and a large fraction of agentic projects are expected to be cancelled. We argue that this is substantially a measurement problem: public benchmarks answer "what can the model do?", whereas a deployment decision requires "is this workflow fit, reliable, safe and worth scaling - here, on our data, under our controls?". We present EnterpriseVal, a use-case-level evaluation system that closes this gap. It comprises (i) a formal specification of the use case and of the frozen socio-technical configuration under test, model, prompts, retrieval, tools, guardrails and human oversight, with an autonomy level and consequence tier that jointly set the required evaluation intensity; (ii) a metric catalogue spanning fidelity, utility, efficiency, reliability, assurance and oversight; (iii) a grading protocol that scales blinded expert judgement with calibrated LLM-as-judge scoring through prediction-powered inference; (iv) a two-tier threshold gate, stated as an executable algorithm, that maps metric vectors with confidence bounds to REJECT/CONDITIONAL/SCALE decisions; and (v) a value-and-risk model in which the reviewer catch rate is a measured parameter. We report a pilot across three workflows in a global bank. In credit-memo drafting, human-graded citation precision reached 88% and hallucination rate 1.6% for the best model against gates of 70% and 5%; in procedure transformation, analyst refinement effort fell from an estimated 27.4 to 2.9 hours per document. We separate established results, documented pilot evidence, the proposed system and open hypotheses, and specify the experiments required for full validation