A BERT-based Question Answering Architecture for Spanish Language

Abstract

QA systems have had various approaches to achieve their goal of solving naturally formed questions, recent works use state of the art techniques such as neural networks, QA systems in different languages are increasing, as evidenced, they are advancing at different rates, despite the fact that there are efforts to increase research in this type of systems. In this research we analyze the main aspects of the contributions to Question Answering and present an architecture that is capable of answering questions in Spanish. The initial question is received by the system, which may or may not have a document corpus from which to extract the answer, if it does not have a specified document corpus, the Answer Generation module returns the answer to the initial question. The purpose of the system is to provide answers to factoid questions posed by users through a web and mobile platform. BI-LSTM was used for document retrieval and BERT was used to generate the answers. We tested the architecture with ten thousand questions reaching an accuracy of 0.7856. The result improved by entering QA to a more specialized BERT model adapted for the Spanish language, the multilingual version of BERT and the Spanish version of BETO were used.