AI summaryⓘ
The authors studied how measuring AI and Industry 4.0 readiness with one overall score can hide differences in opinions about specific technologies and applications. Using a survey with nearly 1,000 respondents evaluating 17 AI and robotics challenges, they found a lot of disagreement even on the same challenges. Most variation in scores came from differences between people and individual responses rather than differences between challenge types or sectors. They found that readiness scores vary across applications within sectors and between respondents with different backgrounds. The authors suggest readiness reports should keep detailed application-level data and explain whose views are included instead of only giving one summary score.
AI readinessIndustry 4.0survey analysisvariance decompositiontechnology assessmentreadiness scoreapplication variationrespondent differencesrobotics challengesevaluation metrics
Abstract
AI and Industry 4.0 readiness assessments often summarise preparedness using a single score for an organisation, application domain or sector. Those summaries can conceal disagreement about the same technology and variation among applications grouped under one sector label. We test how much information is lost through this aggregation using a card-based survey in which 982 respondents provided 15,200 readiness evaluations across 17 named AI and robotics challenges. Readiness is perceived community preparedness and available resources, not personal willingness or audited organisational capability. Respondents frequently disagreed about identical challenges, with card-level readiness standard deviations of $1.03$-$1.26$ on a five-point scale. A crossed decomposition attributes 32.7% of observed variation to stable respondent differences, 7.3% to differences among challenges, and 60.0% to response-level variation that also contains measurement error. Differences among challenge-family means account for only about 2% of variation, with substantially more variation among people, applications and person-family judgements. Manufacturing has the highest mean readiness, yet shop-floor robotics, process-optimisation AI and general decision-support applications are judged differently. Computer-science and AI/ML respondents report higher readiness than non-technical respondents across challenge families, whereas engineering respondents do not report higher Manufacturing readiness. Sector rankings are therefore best used as portfolio summaries rather than evidence that an industry is uniformly ready or behind. Readiness reporting should retain application-level disagreement, disclose whose judgements form the average, and consider ethics, cyber security, literacy and capability needs without collapsing them into a single score.