Improving zero-shot parsing for low-resource languages using bootstrapping
Zero-shot Dependency Parsing with Unsupervised Cross-Lingual Bootstrapping
Computation and Language
Summary
Parsing sentences to understand their structure is hard, especially for languages with little training data. The authors propose a method that improves how language models learn syntax across languages without needing labeled examples. They use an unsupervised bootstrapping technique to help the model better recognize sentence structure in new languages. This approach led to better parsing performance for languages with limited resources.
What this means in practice
- •For machine translation engineers: Improve the understanding of sentence structures in low-resource languages to enhance translation accuracy without labeled syntactic data.
- •For chatbot developers: Build chatbots that better grasp user inputs in many languages by improving their syntactic parsing with zero-shot methods.
Authors
Lalita Lowphansirikul, Attapol Rutherford, Jian Gang Ngui, Sarana Nutanong, Peerat Limkonchotiwat
Abstract
Pre-trained language models (PLMs) with encoder-based architectures have shown impressive capabilities in zero-shot cross-lingual transfer for various language understanding tasks. However, applying this technique to dependency parsing remains a significant challenge due to its syntactic nature. To boost model generalizability across linguistic typologies, we propose a cross-lingual unsupervised bootstrapping method to improve syntactic knowledge within the PLM. We show that our method achieves a significant improvement in zero-shot parsing performance in low-resource languages. Analysis of these bootstrapped models uncovers increased robustness in recognizing syntactic structures, evidenced by higher scores in parameter-free tree probing tests.