Team dina at SemEval-2022 Task 8: Pre-trained Language Models as Baselines for Semantic Similarity

Abstract

This paper describes the participation of the team "dina" in the Multilingual News Similarity task at SemEval 2022. To build our system for the task, we experimented with several multilingual language models which were originally pre-trained for semantic similarity but were not further fine-tuned. We use these models in combination with state-of-the-art packages for machine translation and named entity recognition with the expectation of providing valuable input to the model. Our work assesses the applicability of such "pure" models to solve the multilingual semantic similarity task in the case of news articles. Our best model achieved a score of 0.511, but shows that there is room for improvement.

Related Papers

• → Mahtab at SemEval-2017 Task 2: Combination of Corpus-based and Knowledge-based Methods to Measure Semantic Word Similarity(2017)8 cited
• → JBNU at SemEval-2020 Task 4: BERT and UniLM for Commonsense Validation and Explanation(2020)3 cited
• → What is SemEval evaluating? A Systematic Analysis of Evaluation Campaigns in NLP(2021)2 cited
• System description of Semantic Textual Similarity (STS) in the SemEval-2012 (Task 6)(2012)
• Tiantianzhu7:System Description of Semantic Textual Similarity (STS) in the SemEval-2012 (Task 6)(2012)