Abstract
Natural language processing (NLP) has become ubiquitous in our lives and models are frequently tasked with problems ranging from machine translation to named entity recognition.Modern models may support hundreds of languages, but most remain impractical due to insufficient training data.
This often results in problems where models produce "confidently incorrect" outputs which make them even less useful for less-resourced languages.
Even when a model does work for a given LRL, small test sets also complicate evaluation as observed results may involve substantial uncertainty.
In this thesis, we investigate how these effects of data scarcity can be mitigated by incorporating various forms of prior information into the NLP pipeline.
For machine translation, we investigate the use of morphologically-inspired tokenization as a source of linguistic information and find, contrary to previous literature, that morphological plausibility of tokens is less important than previously thought. In contrast, our experiments show that even randomized segmentation can yield similar results.
In addition, we investigate the utility of data augmentation and language model priors and find that both can yield significant performance improvements when training low-resource MT models from scratch.
On the dataset side, we introduce ParaNames, a massively multilingual resource of parallel entity names, whose utility we demonstrate as a gazetteer for low-resource NER, as well as as synthetic data source for LLM-based MT.
We also illustrate how uncertainty in results can be accurately estimated in multilingual NLP evaluations and propagated to downstream leaderboard rankings using both a nonparametric resampling procedure as well as parametric Bayesian models.