Using Text-Based Causal Inference to Disentangle Factors Influencing Online Review Ratings
2026-06-02 • Computation and Language
Computation and Language
AI summaryⓘ
The authors studied how different parts of online reviews affect the overall rating of a product or service. They used a method called CausalBERT to figure out the impact of specific aspects, like school administration, on ratings while handling the tricky overlaps between these aspects. They improved CausalBERT by making its predictions more accurate and easier to understand. Testing on over 600,000 school reviews, they found that views on administration and test performance strongly influence overall school ratings. Their work helps better separate the effect of each review aspect on overall opinions.
aspect-based sentiment analysiscausal analysisCausalBERTtemperature scalingconfound adjustmenthyperparameter optimizationinterpretabilityonline reviewstreatment assignmentsemi-synthetic data
Authors
Linsen Li, Aron Culotta, Nicholas Mattei
Abstract
Online reviews provide valuable insights into the perceived quality of facets of a product or service. While aspect-based sentiment analysis has focused on extracting these facets from reviews, there is less work understanding the impact of each aspect on overall perception. This is particularly challenging given correlations among aspects, making it difficult to isolate the effects of each. This paper introduces a methodology based on recent advances in text-based causal analysis, specifically CausalBERT, to disentangle the effect of each factor on overall review ratings. We enhance CausalBERT with three key improvements: temperature scaling for better calibrated treatment assignment estimates; hyperparameter optimization to reduce confound overadjustment; and interpretability methods to characterize discovered confounds. In this work, we treat the textual mentions in reviews as proxies for real-world attributes. We validate our approach on real and semi-synthetic data from over 600K reviews of U.S. K-12 schools. We find that the proposed enhancements result in more reliable estimates, and that perception of school administration and performance on benchmarks are significant drivers of overall school ratings.