Interdisciplinary Contributions to Methods

Detecting and Countering LLM Pollution
Ensuring Rigor in AI-Driven Behavioral Science with Standardized Reporting

The Center is decisively neither a traditional computer science department nor a traditional behavioral science department. Rather, it brings together scientists from diverse disciplines in order to shed light on the phenomena of interest (see Figure 1). In particular, CHM employs scientists from three major groups of disciplines. First, computer scientists and data scientists provide the essential technical capability to produce the computational systems the Center is interested in studying (e.g., a reinforcement learning algorithm or a fine-tuned LLM generative adversarial network) and to create technical measurement instruments (e.g., to collect social media posts from X or to scrape online discussion forums and apply natural language processing techniques to them). However, the primary objective is not to contribute to the field of computer science directly in terms of new computational tools—although this does happen on occasion. This is evident in the fact that the Center's primary publication venues are not computer science conferences and journals. The second pillar of CHM are the social (behavioral) and cognitive sciences, which provide experimental methods and the theoretical foundation for understanding how humans interact with machines. The Center thus aims to hire highly qualified quantitative behavioral scientists from fields as diverse as psychology, political science, economics, biology, sociology, and anthropology. Finally, the disciplines of physics and mathematical/statistical modeling provide an additional set of tools typically not available to the average computer scientist. This enables us to use tools from network science, dynamical systems, differential equations, and multilevel statistical modeling/Bayesian inference.

Detecting and Countering LLM Pollution

This project highlights how CHM's strong interdisciplinary focus gives rise to methodological insights that emerge serendipitously from substantive research questions, which we then examine through careful methodological scrutiny. In this specific case, our experience conducting high-fidelity human–machine interaction research enabled us to recognize response patterns that suggested systematic involvement of large language models (LLMs) across multiple online behavioral studies. These observations were explored through "LLM Pollution," defined as an increasing involvement of LLMs in online behavioral studies, along three interrelated variants: Partial LLM Mediation and Full LLM Delegation lie on a continuum of increasing automation, ranging from participants using models for translation, brainstorming, or rewriting, to cases in which participants only provide instructions to an agentic LLM system that completes the study on their behalf, without further human oversight. Meanwhile, LLM Spillover describes second-order effects of these two variants that might materialize in behavioral changes in response to participants (potentially even those who do not use LLMs themselves) expecting LLM to be increasingly involved in online studies. For example, participants may behave more selfishly if they suspect an "LLM partner" is involved, or they may try to signal to researchers that they are genuinely human.It is already clear that LLM Pollution presents us with an arms-race where increasingly sophisticated systems adapt continuously to mitigation strategies that researchers and online platforms may deploy, which may range from behavioral tracking over targeted detection items to post hoc analyses. This makes it all the more important to raise attention to this issue; to leverage collaborative and cumulative empirical testing of both threats and mitigation strategies; and to form a multilayered community response to it. As a first step in this direction, the outlined concept of LLM Pollution, its variants, and exemplary mitigation strategies have been presented in a forward-looking conceptual framework (Rilla et al., 2026). As LLM Pollution is poised to undermine the epistemic grounding of online research on human cognition and behavior, it is anticipated that this contribution will help catalyze a broader methodological conversation and coordinated community response.

Key Reference

Ensuring Rigor in AI-Driven Behavioral Science with Standardized Reporting

How can behavioral science responsibly use large language models without sacrificing transparency, reproducibility, and ethical accountability?

LLMs offer groundbreaking opportunities for the behavioral and social sciences, from simulating human decision-making in silico to conducting large-scale data annotation. However, their rapid evolution and lack of transparency threaten research rigor. Because proprietary models change without notice, and minor configuration differences can completely alter outputs, the field faces an urgent need for standardized reporting.

To address these challenges, an international coalition of experts developed GUIDE-LLM: Guidelines for the Use of LLMs in Behavioral and Social Science (Feuerriegel et al., 2026). This consensus-based, 14-item checklist establishes a minimum reporting standard to promote transparency, reproducibility, and ethical accountability.

The framework targets critical sources of variability in LLM research across seven core domains. It asks researchers to clearly define the scope of LLM use and whether systems were fully automated or human-in-the-loop; disclose the model and system details including provider, version ID, size, and access mode; report configuration parameters such as temperature, seeds, and maximum token limits; provide exact prompts and relevant system instructions; document the processing of personal or sensitive data, validation protocols, post-processing steps as well as the underlying code, scripts, and competing interests.

Developed using a rigorous Delphi consensus process, GUIDE-LLM intentionally condenses an initial pool of 29 items into a concise, 14-item baseline to prevent administrative burden while remaining highly effective. Maintained as a "living document," the framework is designed to adapt alongside evolving multimodal systems and agentic AI architectures. Optional checklist items are also available for context-specific issues such as training-data leakage, bias assessment, ethical implications, and computational resource use.

Rather than prescribing specific methodological choices, GUIDE-LLM empowers researchers, editors, and reviewers with a flexible tool to evaluate research quality. By formalizing these documentation standards, the behavioral science community can responsibly leverage AI without sacrificing scientific integrity.

Key Reference