Cross-Domain, Multi-Task Data-to-Text Generation without In-Domain Training Data
2026-08-24 • Computation and Language
Computation and LanguageArtificial Intelligence
AI summaryⓘ
The authors study how to turn different kinds of structured data, like tables and graphs, into text without using examples from the target domain. They compare three methods: zero-shot use of large models, fine-tuning on other data, and a technique called data-driven knowledge distillation (DDKD). Their experiments across five different tasks show that DDKD works best and can even beat bigger models. They also propose a way to improve DDKD by changing the data structure to help the models learn better. Finally, they create an expanded dataset to test their ideas and find that their augmentation method is more effective than just adding more real data.
data-to-text generationstructured dataknowledge distillationzero-shot learningfine-tuningcross-domainaugmentationtransformer modelsQUINTD dataset
Authors
Yifei Song, Kun Efimov-Zhang, Claire Gardent
Abstract
Structured data exists in many forms (tables, knowledge graphs, charts, and time series), and converting it into text may involve different generation tasks. However, most prior work on data-to-text (D2T) generation has focused on specific tasks and datasets, relying either on task-specific training data or on the zero-shot capabilities of large language models. We study cross-domain D2T generation in a setting where neither in-domain training text nor test references are available, and where domains, generation goals, and input structures vary substantially. We compare data-driven knowledge distillation (DDKD) against zero-shot inference and fine-tuning on out-of-domain D2T data, and introduce structure-preserving augmentation via structural subsampling and perturbation. Experiments on five benchmarks show that, at constant model size (1.7B parameters), DDKD consistently outperforms both fine-tuning and zero-shot inference. Moreover, the resulting small models outperform a much larger finetuned model on two of the five domains, achieving comparable performance on the remaining three. We further construct QUINTD-5, a fivefold extension of QUINTD-1, and show that simply scaling real target-domain inputs yields only modest gains, whereas our augmentation strategy remains more effective and more cost-efficient for cross-domain distillation.