Which argument is more convincing? Analyzing and predicting convincingness of Web arguments using bidirectional LSTM

Citations Over TimeTop 1% of 2016 papers

Abstract

We propose a new task in the field of computational argumentation in which we investigate qualitative properties of Web arguments, namely their convincingness. We cast the problem as relation classification, where a pair of arguments having the same stance to the same prompt is judged. We annotate a large datasets of 16k pairs of arguments over 32 topics and investigate whether the relation "A is more convincing than B" exhibits properties of total ordering; these findings are used as global constraints for cleaning the crowdsourced data. We propose two tasks: (1) predicting which argument from an argument pair is more convincing and (2) ranking all arguments to the topic based on their convincingness. We experiment with feature-rich SVM and bidirectional LSTM and obtain 0.76-0.78 accuracy and 0.35-0.40 Spearman's correlation in a cross-topic evaluation. We release the newly created corpus UKPConvArg1 and the experimental software under open licenses.

Related Papers

• → Introduction to structured argumentation(2014)147 cited
• → Impartation of Argumentation Skills: Impact of Scaffolds on the Quality of Arguments(2017)4 cited
• → Building a System for Finding Objections to an Argument(2012)
• 'A good old argument' : the discursive construction of family and research through argumentation.(1996)