What Knowledge Do We Produce from Social Media Data and How?

HCI and CSCW research that uses social media data to make inferences about individuals and communities has proliferated in the last decade. Previous studies have elaborated on methodological concerns and challenges and examined the assumptions and values underlying knowledge production through quantification and data. We expand this line of research by making visible and explicit the conventions and practices that establish, sustain, and reinforce current discourses in social media research. We conducted a Critical Discourse Analysis on 84 research papers published between 2010 and 2023 in CHI, CSCW and GROUP, that combine social media data and computational methods. Our findings show that plenty of this work legitimizes social media data as a valid source of information by centering its public availability, unobtrusiveness, and volume. Furthermore, to justify computational techniques, these papers prioritize computational expediency over data and method appropriateness. We argue that these embedded strategies may result in a methodological and epistemological distance between researchers and the studied communities, impacting problem framing, data collection, and findings application. With this work, we join the voices that have advocated for increased reflexivity in HCI and CSCW communities to scrutinize knowledge production and the role of researchers as knowledge producers.