Developing a successful SemEval task in sentiment analysis of Twitter and other social media texts

Citations Over TimeTop 1% of 2016 papers

Abstract

We present the development and evaluation of a semantic analysis task that lies at the intersection of two very trendy lines of research in contemporary computational linguistics: (1) sentiment analysis, and (2) natural language processing of social media text. The task was part of SemEval, the International Workshop on Semantic Evaluation, a semantic evaluation forum previously known as SensEval. The task ran in 2013 and 2014, attracting the highest number of participating teams at SemEval in both years, and there is an ongoing edition in 2015. The task included the creation of a large contextual and message-level polarity corpus consisting of tweets, SMS messages, LiveJournal messages, and a special test set of sarcastic tweets. The evaluation attracted 44 teams in 2013 and 46 in 2014, who used a variety of approaches. The best teams were able to outperform several baselines by sizable margins with improvement across the 2 years the task has been run. We hope that the long-lasting role of this task and the accompanying datasets will be to serve as a test bed for comparing different approaches, thus facilitating research.

Related Papers

• → SemEval-2007 task 07(2007)155 cited
• → SIEL: Aspect Based Sentiment Analysis in Reviews(2015)20 cited
• → What is SemEval evaluating? A Systematic Analysis of Evaluation Campaigns in NLP(2021)2 cited
• → NAYEL at SemEval-2020 Task 12: TF/IDF-Based Approach for Automatic Offensive Language Detection in Arabic Tweets(2020)
• → NAYEL at SemEval-2020 Task 12: TF/IDF-Based Approach for Automatic\n Offensive Language Detection in Arabic Tweets(2020)