School of Information Studies Associate Professor Lu Xiao has been recognized for research that employs multiple methodologies to examine the factors surrounding a comment’s persuasive power on social media, and the ability of those comments to change viewpoints that have been expressed online.
Xiao and co-author Taraneh Khazaei, of Microsoft in Toronto, were chosen for the “Best Methods Paper” award at the 10th International Social Media & Society Conference held in Toronto recently. Their paper, “Changing Others’ Beliefs in Social Media: Online Comments’ Persuasiveness,” was selected for its comprehensive analysis methods and the variety of analyses used in evaluating central questions around the factors that influence the persuasiveness of comments posted online.
The research involves discussion comments posted on a sub-Reddit forum, “Change My View.” It explores different dimensions of the language used, the structure of the comments made, attributes of users, and other factors that contribute to a comment’s persuasive value, according to Xiao. Specifically, she says, the examination looked at these aspects of the conversations:
- The relevance between the comment and the view it changed
- The relative position of the comment in the online discussion
- The psychological attributes (emotional tone) of the comment
- The use of function words
- The level of writing sophistication and the comprehensibility of the comment
- The status of the commenter in the online environment.
The research project also investigates the reasons why online users were persuaded to change their opinions. The analysis uses an online natural language processing tool to look at the specific language used, but also takes into account several other aspects of the discussion. Findings were linked to earlier research on persuasion and belief change in traditional forms to seek to uncover when and how those factors apply to online persuasion in conversations.
“We looked at the inference of comments, their order of discussion, and at users’ credibility scores. We also did a statistical analysis when comparing comments of different groups. We measured the relevance between the original comment submission and subsequent comments, then sorted the comments on the original post based on how relevant they were. That ranking was then used for further analysis,” Xiao notes.
There has been an increasing interest in studying online persuasion, notes Xiao. “We hope to examine whether there are indicators and features that we can identify from those online users’ digital tracers and the online environments that can signify whether a comment is persuasive. If we can do that, we can improve the tool we’ve developed to signal whether a particular comment is powerful and whether it is persuasive or not persuasive,” she says. The persuasion analysis tool is available to the public online at https://persuasiontool.herokuapp.com/.
The practical value of the study comes from the insights it can yield into how to write effectively in the online environment, and whether writing effectively is the same or different when the information is conveyed in online and offline environments, Xiao says. While the analysis presented in this paper is useful to assess persuasiveness in this situation, it applies only to the particular discussions referenced and as they occurred on the Reddit platform. Other conversations on other social platforms may present different comparisons, she adds. Previously, Xiao’s research looked at online persuasion factors in Wikipedia discussions. Next, she hopes to examine other online environments and platforms, such as the comments left on the review site, Yelp.
The conference is an annual gathering of leading social media researchers from around the world. Organized by the Social Media Lab at Ted Rogers School of Management at Ryerson University, the conference showcases research from scholars working in many fields, including management, communication, computer science, education, journalism, information science, political science, and sociology.