Open platform experiments expand testing on social media sites
Open Platform Field Experiments: Expanding the Design Space of Experimental Research on Social Media
Computers and SocietyHuman-Computer InteractionSocial and Information Networks
Summary
It is hard for researchers outside social media companies to run experiments that show cause and effect directly on popular platforms. The authors explain a new way to do this by using open social media platforms that let researchers change important parts like how content is recommended or moderated. They call these Open Platform Field Experiments (OPFEs) and show how OPFEs offer new opportunities that other methods don’t have. The paper shows examples using Bluesky and the AT Protocol and describes how to run these experiments from start to finish.
What this means in practice
- •For social media platform developers: Create experimental features that allow outside researchers to safely test and measure effects on live social media interactions.
- •For online community managers: Use open platform experiments to evaluate impact of moderation and recommendation changes on user behavior in real environments.
A position paper. It proposes an approach and reports no results.
Authors
Jordi Guillem Condom-Tibau, Giovanni Puccetti, Clara Bacciu, Matteo Abrate, Stefano Cresci
Abstract
Despite a growing demand for causal evidence about social media, independent researchers remain severely constrained in their ability to conduct experiments directly on online platforms. To cope, multiple methodological workarounds have emerged - from controlled surveys and simulations to client-side overlays and platform partnerships - each requiring distinct trade-offs between desirable experimental properties. The recent emergence of open social media platforms offers a qualitatively different methodological opportunity. Here we propose a design space of social media experimentation and discuss Open Platform Field Experiments (OPFEs). OPFEs represent a distinct class of experimental approaches that enable independent researchers to directly intervene on functional platform components - such as clients, recommendation systems, and moderation services - within live social media environments. Through a comparative analysis of experimental archetypes, we show that OPFEs occupy a previously unexplored region of the design space. We then bridge theory and practice by characterizing the architectural and governance elements that enable OPFEs, mapping them onto Bluesky and the AT Protocol, and illustrating the end-to-end lifecycle of a complete OPFE design. Overall, this work establishes OPFEs as a practical methodological paradigm for independent, transparent, and ecologically grounded experimentation on open social media.