publications

2024

1. 

   Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning

   Shivalika Singh, Freddie Vargus, Daniel Dsouza, and 30 more authors

   2024

   Abs

   Datasets are foundational to many breakthroughs in modern artificial intelligence. Many recent achievements in the space of natural language processing (NLP) can be attributed to the finetuning of pre-trained models on a diverse set of tasks that enables a large language model (LLM) to respond to instructions. Instruction fine-tuning (IFT) requires specifically constructed and annotated datasets. However, existing datasets are almost all in the English language. In this work, our primary goal is to bridge the language gap by building a human-curated instruction-following dataset spanning 65 languages. We worked with fluent speakers of languages from around the world to collect natural instances of instructions and completions. Furthermore, we create the most extensive multilingual collection to date, comprising 513 million instances through templating and translating existing datasets across 114 languages. In total, we contribute four key resources: we develop and open-source the Aya Annotation Platform, the Aya Dataset, the Aya Collection, and the Aya Evaluation Suite. The Aya initiative also serves as a valuable case study in participatory research, involving collaborators from 119 countries. We see this as a valuable framework for future research collaborations that aim to bridge gaps in resources.

2022

1. 

   NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation

   Kaustubh D. Dhole, Varun Gangal, Sebastian Gehrmann, and 123 more authors

   2022

   Abs

   Data augmentation is an important component in the robustness evaluation of models in natural language processing (NLP) and in enhancing the diversity of the data they are trained on. In this paper, we present NL-Augmenter, a new participatory Python-based natural language augmentation framework which supports the creation of both transformations (modifications to the data) and filters (data splits according to specific features). We describe the framework and an initial set of 117 transformations and 23 filters for a variety of natural language tasks. We demonstrate the efficacy of NL-Augmenter by using several of its transformations to analyze the robustness of popular natural language models. The infrastructure, datacards and robustness analysis results are available publicly on the NL-Augmenter repository

2. 

   GEMv2: Multilingual NLG Benchmarking in a Single Line of Code

   Sebastian Gehrmann, Abhik Bhattacharjee, Abinaya Mahendiran, and 74 more authors

   2022

   Abs

   Evaluation in machine learning is usually informed by past choices, for example which datasets or metrics to use. This standardization enables the comparison on equal footing using leaderboards, but the evaluation choices become sub-optimal as better alternatives arise. This problem is especially pertinent in natural language generation which requires ever-improving suites of datasets, metrics, and human evaluation to make definitive claims. To make following best model evaluation practices easier, we introduce GEMv2. The new version of the Generation, Evaluation, and Metrics Benchmark introduces a modular infrastructure for dataset, model, and metric developers to benefit from each others work. GEMv2 supports 40 documented datasets in 51 languages. Models for all datasets can be evaluated online and our interactive data card creation and rendering tools make it easier to add new datasets to the living benchmark.

2020

1. 

   Data Augmentation Techniques for Tabular Data

   Abinaya Mahendiran, and Vedanth Subramaniam

   2020

   PDF

2019

1. 

   Machine Learning & IT Operations to Manage End-to-End Machine Learning Life Cycles

   Jai Ganesh, Archisman Majumdar, Saurabh Singh, and 2 more authors

   2019

   PDF

2018

1. 

   Improving Search Relevance in Question Answering Engine

   Abinaya Mahendiran, and Faustina Selvadeepa

   2018

   PDF

2015

1. 

   Cross-Referencing Cancer Data Using GPU for Multinomial Classification of Genetic Markers

   Abinaya Mahendiran, Srivatsan Sridharan, Sushanth Bhat, and 1 more author

   CSI Journal of Computing, Jun 2015