Fine-Tuned Transformer Models for Question Answering

Citations Over Time

Abstract

Effective question-answering is crucial for applications involving natural language processing such as conversational artificial systems. The pre-trained BERT (Bidirectional Encoder Representations from Transformers) model achieved outstanding results on the Stanford Question Answering Dataset (SQuAD), a well-known benchmark for question answering tasks. The three different BERT models (BERT-base, BERT-large, DistilBERT) are pre-trained on general text corpus. These BERT models are trained to identify the right context in a paragraph to give answers. Further, fine-tuning the models with few-shot learning on domain related questions and answers improves the performance. Hence, our work proposes different strategies to improve the performance of BERT models on tasks requiring domain-specific question-answering tasks using a customized SQuAD dataset. Our work highlights the significance of optimizing the BERT models on domain-specific data for enhanced performance in particular tasks. The results of this study have implications for domain-specific knowledge in natural language processing. With a score of 0.9632, the BERT-Large model has the best accuracy out of all the BERT models tested. The performance of the BERT-Large model consistently beats that of the other models in several parameters, including precision, recall, and F1 score.

Related Papers

• → Fine-Tuned Transformer Models for Question Answering(2023)2 cited
• Question answering systems: a partial answer(2007)
• → Similarity Detection of Natural-Language Questions and Answers using the VANiLLa dataset(2021)2 cited
• → A Comparative Study of Transformer-Based Language Models on Extractive Question Answering(2021)22 cited
• → Exploring Neural Net Augmentation to BERT for Question Answering on SQUAD 2.0(2019)1 cited