Assessing the Veracity of Climate Change Information on Twitter

Assessing the Veracity of Climate Change Information on Twitter Track 1: Text Similarity Approach Abstract This research investigates the dissemination of false and misleading information related to climate change on social media, while examining key contributors and influencers responsible for propagating such deceptive content. Disinformation on social media platforms poses a challenge as it is often presented in a complex and convoluted manner, making it difficult to discern. Furthermore, entities with intentions to deceive may collaborate with other organizations to amplify their messages. Our study analyzes climate change-related tweets from 2008 to 2023, utilizing machine learning algorithms to identify and categorize misleading claims based on the CARDS system of climate change misinformation super-claims proposed by Coan et al., 2021. We employ a two-way coding method, comparing cosine similarity with human coding, and achieve approximately 75% overlap between the techniques. We also examine the Twitter activity of organizations funded by ExxonMobil and scrutinize their messaging to identify narratives. Additionally, we pinpoint prominent Twitter accounts that frequently post misinformation and investigate the accounts that propagate this content via retweets, employing a network analysis approach. Background In today's digitally connected world, social media platforms have become integral to the dissemination of information, shaping public opinion and discourse on various subjects, including climate change. However, the vast and unregulated nature of these platforms has led to a proliferation of misinformation, with climate change being no exception. As this global crisis intensifies, the spread of misleading and false information on social media has the potential to undermine scientific consensus, hinder public understanding, and stall necessary policy action. It is crucial to identify and combat the sources and spread of climate change misinformation on social media to foster informed decision-making and encourage meaningful action. This research aims to investigate the dynamics of misinformation dissemination on social media, scrutinize the key contributors and influencers perpetuating such falsehoods, and provide insights into the narratives and strategies employed to mislead the public on the issue of climate change. TEMPLATE DESIGN © 2007 www.PosterPresentations.com Track2: Machine Learning Approach with CARDS We hypothesized that tweets of which semantics are highly similar to a misinformation claim likely contain the claim in texts. To measure semantic similarity between tweets and claims, we converted all tweets and simplified claims into embedding vectors by using Google's T5 model. This model was pre-trained on the extensive C4 dataset, comprising over 750GB of cleaned and filtered web text from the Common Crawl archive, which allowed T5 to develop a rich understanding of language patterns and structures. The model was subsequently fine-tuned on various downstream NLP tasks, resulting in state-of-the-art performance across numerous benchmarks (Raffel et al., 2020). The CARDS (Climate Advocacy, Research, and Discourse Systems) model is a novel approach for systematically categorizing and analyzing climate change discourse based on a hierarchical taxonomy of contrarian claims and counterclaims. By combining machine learning with expert annotators, the model classifies over 32,000 paragraphs of text from various sources, such as contrarian think tank reports, into super-claims and sub-claims. This comprehensive and fine-grained analysis of climate discourse allows researchers and policymakers to better understand the dynamics of the climate debate and develop effective communication strategies. https://www.oreilly.com/library/view/statistics-for-machine/9781788295758/eb9cd609-e44a-40a2-9c3a-f16fc4f5289a.xhtml Google T5 sentence embeddings The Google T5 model is a versatile and powerful NLP model based on the transformer architecture, excelling in various tasks like translation, summarization, question-answering, and sentiment analysis. By using a unified text-to-text input-output format and leveraging the C4 dataset for pre-training, T5 achieves state-of-the-art results across multiple benchmarks, making it an effective choice for language understanding and generation challenges. Coan et al., 2021 How do we leverage CARDS? We applied the CARDS model to a dataset of 680k 2007 to 2023, categorizing them into 17 sub-claims. Academic API and search queries, we gathered 1.44 leveraged the original CARDS training set to rebuild the performance on this social media dataset. cleaned tweets from Using the Twitter million tweets and model and assess its Raffel et al., 2020 Results we computed the cosine similarity between the embeddings of simplified claims and tweets. We selected the top 20 tweets with the highest cosine similarity scores in each subclaim to create an initial version of the survey. For example: Superclaim subclaims Keywords for collecting the tweets anchors tweets Cosine similarity Global Heading into warming is not ice age happening global warming heading (head OR ice age heading) ice age into Somebody 0.8942 said something about global warming. Feels like ice age is coming #FreezingCold https://t.co/ItY3 yP6JfL Climate Politics are solutions won’t harmful work climate (policy OR policies) (harm OR harmful) are Some policies 0.8760 to fight climate change have done more harm than good http://t.co/HFB uVXNDrz policies harmful Initial Results To assess the prediction performance, we chose a sample of 30 tweets from each group, resulting in 540 tweets. We employed the majority rule to label our data, assigning three labels per tweet. So far, we have successfully manually labeled 150 tweets, achieving approximately 60% accuracy.