Artykuł w czasopiśmie
Brak miniatury
Licencja
CC-BY-NC-NDCC-BY-NC-ND - Uznanie autorstwa - Użycie niekomercyjne - Bez utworów zależnych
 

Pseudo-labeling with transformers for improving Question Answering systems

Uproszczony widok
dc.abstract.enAdvances in neural networks contributed to the fast development of Natural Language Processing systems. As a result, Question Answering systems have evolved and can classify and answer questions in an intuitive yet communicative way. However, the lack of large volumes of labeled data prevents large-scale training and development of Question Answering systems, confirming the need for further research. This paper aims to handle this real-world problem of lack of labeled datasets by applying a pseudo-labeling technique relying on a neural network transformer model DistilBERT. In order to evaluate our contribution, we examined the performance of a text classification transformer model that was fine-tuned on the data subject to prior pseudo-labeling. Research has shown the usefulness of the applied pseudo-labeling technique on a neural network text classification transformer model DistilBERT. The results of our analysis indicated that the model with additional pseudo-labeled data achieved the best results among other compared neural network architectures. Based on that result, Question Answering systems may be directly improved by enriching their training steps with additional data acquired cost-effectively.
dc.affiliationUniwersytet Warszawski
dc.conference.countryPolska
dc.conference.datefinish2021-09-10
dc.conference.datestart2021-09-08
dc.conference.placeSzczecin
dc.conference.seriesInternational Conference on Knowledge-Based and Intelligent Information and Engineering Systems
dc.conference.seriesInternational Conference on Knowledge-Based and Intelligent Information and Engineering Systems
dc.conference.seriesshortcutKES
dc.conference.shortcutKES 2021
dc.conference.weblinkhttp://kes2021.kesinternational.org/index.php
dc.contributor.authorKuligowska, Karolina
dc.contributor.authorKowalczuk, Bartłomiej
dc.date.accessioned2024-01-25T18:38:21Z
dc.date.available2024-01-25T18:38:21Z
dc.date.copyright2021-10-02
dc.date.issued2021
dc.description.accesstimeAT_PUBLICATION
dc.description.financePublikacja bezkosztowa
dc.description.versionFINAL_PUBLISHED
dc.description.volume192
dc.identifier.doi10.1016/J.PROCS.2021.08.119
dc.identifier.issn1877-0509
dc.identifier.urihttps://repozytorium.uw.edu.pl//handle/item/117526
dc.identifier.weblinkhttps://www.sciencedirect.com/science/article/pii/S1877050921016082
dc.languageeng
dc.pbn.affiliationeconomics and finance
dc.relation.ispartofProcedia Computer Science
dc.relation.pages1162-1169
dc.rightsCC-BY-NC-ND
dc.sciencecloudnosend
dc.subject.enNatural Language Processing
dc.subject.enQuestion Answering systems
dc.subject.enpseudo-labeling
dc.subject.enneural networks
dc.subject.entransfer learning
dc.subject.enknowledge distillation
dc.titlePseudo-labeling with transformers for improving Question Answering systems
dc.typeJournalArticle
dspace.entity.typePublication