PhysicsMate improves Bengali physics question answering accuracy
PhysicsMate: A Curriculum-Grounded Bengali Benchmark for Secondary Physics QA with Small-Model Adaptation
Computation and Language
Summary
Physics problems in Bengali secondary schools are hard for general AI because they need specific terms, units, and steps from the curriculum. The authors created PhysicsMate, a set of 1834 questions based directly on the Bengali Grade 9-10 physics syllabus and linked to structured knowledge about physical concepts. They adapted language models of different sizes using this data, improving their ability to answer questions correctly, especially for knowledge about physical quantities and laws. The biggest adapted model can run offline on small computers, helping physics education in places with limited internet and resources.
What this means in practice
- •For educational software developers: Create offline physics tutoring apps for Bengali secondary students that use curriculum-specific language and knowledge to improve accuracy.
- •For mobile device engineers: Deploy quantized language models on low-resource devices for local physics question answering without continuous internet access.
Authors
Rashid Azraf Jahin, Saadman Sajid, Khan Raiyan Ibne Reza, Sumaiya Tabassum Nimi
Abstract
Bengali secondary education lacks curriculum-grounded benchmarks for STEM question-solving, and general-purpose language models struggle with the precise terminology, unit conventions, and derivations that physics problems demand. We introduce PhysicsMate, a benchmark of 1834 question-answer pairs built from the National Curriculum and Textbook Board (NCTB) Grade 9-10 physics syllabus and grounded in a multi-relational knowledge graph of 1760 nodes and 2600 edges across ten ontological types. We Low-Rank Adapt at 0.6B, 1.7B, and 4B parameters, with a unified recipe and demonstrate a significant increase in closed-book accuracy in all scales (+5.5, +15.0, and +23.3 percentage points). A node-type analysis shows that the most benefited by adaptation is the structured curricular knowledge, which consists of physical quantities and named laws, while the least benefited is the loosely specified entity-level knowledge. The 4B model has been adapted and quantized to a small offline binary that can be used for local inference in resource constrained environments and offers a viable path to curriculum aligned physics support in environments with limited connectivity and hardware.