publications

2026

1. BabyLM

   Can Translation Bridge the Data Gap for Low-Resource Languages? A Case Study of Hindi and Telugu

   Venkata Naga Sai Vishnu Rohit Pulipaka and Libby Barak

   In Proceedings of the Second BabyLM Workshop, pp. 279–288, Budapest, Hungary. Association for Computational Linguistics, Oct 2026

   AbsHTMLPDFDOICodeHindi dataTelugu dataBib

   Low-resource languages often lack sufficient training data for developing small language models, limiting the application of recent language acquisition benchmarks beyond English. Moreover, comparison across languages is made harder given differences in training data composition and quality. We investigate whether machine translation can be used to create BabyLM-scale training corpora for low-resource languages by translating the English BabyLM corpus into Hindi and Telugu using IndicTrans2. We train GPT-BERT and GPT-Wee models and evaluate them using perplexity and linguistic benchmarks. GPT-BERT consistently outperforms GPT-Wee across both monolingual and bilingual settings. Although perplexity is lower on in-domain test data, this pattern is consistent across English and the translated languages, suggesting that the effect reflects domain similarity rather than artifacts introduced by translation. Furthermore, the models achieve competitive performance on linguistic evaluations relative to pretrained language models. These findings suggest that translation warrants further analysis to obtain data for developing and evaluating small language models in low-resource settings.

   @inproceedings{pulipaka-barak-2026-translation,
     title     = {Can Translation Bridge the Data Gap for Low-Resource Languages? A Case Study of {H}indi and {T}elugu},
     author    = {Pulipaka, Venkata Naga Sai Vishnu Rohit and Barak, Libby},
     booktitle = {Proceedings of the Second {B}aby{LM} Workshop},
     month     = oct,
     year      = {2026},
     address   = {Budapest, Hungary},
     publisher = {Association for Computational Linguistics},
     url       = {https://aclanthology.org/2026.babylm-main.22/},
     doi       = {10.18653/2026.babylm-main.22},
     pages     = {279--288}
   }

2. arXiv

   Evaluating Dedicated Monolingual and Joint Multilingual Causal Models for Dravidian Languages

   Venkata Naga Sai Vishnu Rohit Pulipaka

   arXiv preprint arXiv:2608.07727, Aug 2026

   AbsarXivPDFCodeBib

   Dravidian languages, mainly Tamil, Telugu, Kannada, and Malayalam make up only a small part of the data used to train multilingual language models, so it's not clear how much per-language ability these models actually keep. I have trained five GPT-2 architecture models from scratch to compare four monolingual models (one each for Tamil, Telugu, Kannada, and Malayalam, each with its own 32K-vocabulary subword tokenizer) against one multilingual model sharing a 64K-vocabulary subword tokenizer across all four languages. All the 5 models are trained on cleaned CC-100, Wikipedia, and Samanantar data. I have tested the models on perplexity, bits-per-byte, tokenizer efficiency, and fine-tuning results which are compared against mGPT. The monolingual models outperform mGPT on sentiment classification and named entity recognition, and their tokenizers proved more efficient than the shared multilingual model across all the languages tested.

   @misc{pulipaka2026dravidian,
     title         = {Evaluating Dedicated Monolingual and Joint Multilingual Causal Models for {D}ravidian Languages},
     author        = {Pulipaka, Venkata Naga Sai Vishnu Rohit},
     year          = {2026},
     eprint        = {2608.07727},
     archivePrefix = {arXiv},
     primaryClass  = {cs.CL},
     url           = {https://arxiv.org/abs/2608.07727}
   }

3. Under review

   Reproducible Fairness Regression Testing for Large Language Model Updates

   Venkata Naga Sai Vishnu Rohit Pulipaka

   Under review, 2026

   AbsCodeBib

   Large language models are updated frequently, but it remains unclear whether newer releases consistently improve in fairness. Most existing evaluations compare models at a single point in time rather than examining how fairness-related behavior changes from one release to the next. In this work, we study these changes through a longitudinal fairness regression framework applied across multiple model families from 4 providers and 7 public bias benchmarks. The evaluation contains 13,300 raw model responses, of which 12,789 remain after filtering. We measured changes in sentiment, toxicity, and stereotype-related behavior while keeping the evaluation procedure fixed across model versions. Across all eight release transitions, sentiment and toxicity show little aggregate change, while the stereotype proxy shows a small increase. At the transition level, 14 of 24 metric comparisons remain significant after Benjamini–Hochberg correction, although most standardized effect sizes are small. Benchmark-level results vary substantially, showing that aggregate trends can hide differences across datasets. The three metrics are also not strongly aligned: per-example correlations remain small, while the largest model benchmark correlation is r = 0.36 between toxicity and stereotype scores. Removing StereoSet and CrowS-Pairs preserves most transition directions, although the strict-transition sentiment and toxicity effects no longer exclude zero. These results support treating fairness evaluation as a regression-testing problem in which model replacements are checked across multiple behavioral dimensions before deployment.

   @unpublished{pulipaka2026fairness,
     title  = {Reproducible Fairness Regression Testing for Large Language Model Updates},
     author = {Pulipaka, Venkata Naga Sai Vishnu Rohit},
     note   = {Under review},
     year   = {2026}
   }

4. Under review

   Script and Morphology Effects on Telugu and Hindi Tokenization Are Tokenizer-Dependent

   Venkata Naga Sai Vishnu Rohit Pulipaka et al.

   Under review, 2026

   Abs

   Hindi has 611 million speakers and Telugu has 96 million speakers. Both languages have limited representation in pre-training data and tokenizer design. This paper examines how 8 tokenizers process formal, informal, native-script, and romanized text in both languages, and includes 144 morphological forms, with 62 Telugu and 82 Hindi samples across 9 language-specific morphological categories. For informal native-script text, Telugu ranges from 2.41 to 22.08 tokens per word, while Hindi ranges from 1.35 to 7.46. The results also show that changing from native to romanized script can greatly change token counts, and some morphological forms are split differently across tokenizers. Overall, tokenization depends on both script and language structure, showing why native and romanized text should be tested separately for Hindi and Telugu.