Tajik text embedder improves finding answers in documents significantly
Saidzoda-Tajik-E5-Large: Fine-Tuning a 560M Embedder for Tajik Retrieval
Computation and Language
Summary
Finding the correct piece of text to answer questions is important, but there was no openly available tool for the Tajik language. The authors adapted a large multilingual language model to better find relevant passages in Tajik laws, tax documents, and Wikipedia by training it on specially created question-and-answer pairs. Their model finds the right passage in the top ten results more often than previous tools, especially on legal and tax texts. While the model performs very well on test questions generated by another model, its advantage is less clear on questions written by people, showing that more human data is needed.
What this means in practice
- •For document search developers: Build or improve Tajik document search systems that retrieve relevant passages for legal and tax questions with higher accuracy.
- •For multilingual nlp engineers: Adapt multilingual language models to low-resource languages like Tajik by fine-tuning with synthetic question-answer pairs including cross-language examples.
Authors
Tohir Saidzoda, Nekruz Maxudov, Bedil Karimov, Abubakr Abdullozoda, Amin Jalilov, Jonibek Kishvarov, Rahmonali Sharipov, Mariyamkhon Shodyeva
Abstract
Finding the passage that answers a question is the first step of most practical systems built on documents, and for Tajik we found no openly described model for it. We present Saidzoda-Tajik-E5-Large, a Tajik text embedder built from multilingual-e5-large-instruct (560M parameters). We asked our own Tajik language model to write questions for passages from Tajik laws, tax documents, Wikipedia and web text, and trained on 235,140 question and passage pairs for one epoch, with hard negatives chosen by another embedder and English and Russian pairs mixed in so the model would not forget those languages. On laws the model had never seen, the right passage is among the top ten results for 93.6% of questions, up from 80.2%; on tax documents, 94.5% against 82.6%. On Belebele, the one test we have that was written by people, the gain is smaller and not confirmed (92.2% to 95.2%, with touching intervals), and Russian and English do not change. On law and tax questions written in the same style as its training questions, it is tied with Giga-Embeddings-3B, a model five times its size; on Belebele the 3B and 8B models score slightly higher, and only after we fine-tuned them on our data are they clearly ahead. We also report what failed: training in bfloat16 broke the model, and replacing Tajik letters with similar Russian ones did not help. Our law, tax and Wikipedia test questions come from the same model that wrote the training questions, so we treat those scores as an upper limit until questions written by Tajik speakers are available.