Summary
Visitor reviews of sacred heritage sites often include details about rituals, cultural rules, and mixed emotions that generic language tools miss. The authors modified a language model system to identify specific sacred heritage topics in reviews of the Pashupatinath Temple in Nepal. They created categories related to temple culture and tested the system on thousands of reviews, checking how well it extracted information about people, places, rituals, and sentiments. Their method showed moderate success and involved human checks to ensure cultural accuracy, but also acknowledged limitations in how broadly it applies.
What this means in practice
- •For cultural heritage analysts: Extract detailed cultural and ritual information from temple visitor reviews using a tailored language model approach.
- •For travel platform developers: Enhance review analysis by incorporating sacred site-specific categories to better represent visitor experiences at religious heritage locations.
Tested on one dataset.
Authors
Hari Sitaula, Manish Sakhakarmy, Pujan Pant, Sagar Newpane
Abstract
Large language model (LLM) pipelines can turn visitor reviews into structured data, but generic schemes may miss rituals, sacred spaces, cultural restrictions and emotionally mixed experiences at living sacred heritage sites. We adapted the LLM-Social Computing (LLM-SC) framework to Pashupatinath Temple, Nepal, by specifying a 12-type entity taxonomy (four original LLM-SC types plus eight sacred-heritage types), four relation types and nine aspect-sentiment dimensions in a GPT-4o extraction prompt, which was applied to a fixed random sample of 2,500 English-language Tripadvisor reviews from a processed corpus of 3,939. The exports contain 11,286 entity, 6,232 aspect-sentiment and 5,590 relation records, covering 2,485, 2,487 and 2,433 reviews, respectively; the eight added sacred-heritage types account for 64.8% of entity records. Against an adjudicated two-annotator reference for 200 reviews, exact normalized surface-and-type matching gave provisional entity scores of 41.0% precision, 21.8% recall and 28.4% micro-F1. Aspect-presence F1 was 53.6% and end-to-end aspect-plus-polarity F1 was 42.6%. Human adjudication supported 153 of 194 first-listed relation candidates (one per review; 78.9%), a candidate-level rate that does not estimate relation recall or whole-graph precision. The contribution is a culturally grounded, domain-specific adaptation with component-wise human evaluation and explicit reproducibility and generalization limitations.