Ballpark: pre-registration

Study. A direct replication of the anchoring task in Many Labs 1 (Klein et al., 2014, Social Psychology 45(3), 142–152), which re-ran Jacowitz and Kahneman (1995).

Registered 2026-10-03, before the first response, by Aramis Lopez. This text is published at /method with its SHA-256 fingerprint and archived on the Wayback Machine before launch.

Question

Do people's estimates move toward an arbitrary number shown just before they answer, and is the effect as large in a self-selected online sample as it was in Many Labs 1?

Hypotheses

For each of the four items, estimates from participants who saw the high anchor are higher than estimates from participants who saw the low anchor:

• H1: distance from San Francisco to New York City
• H2: population of Chicago
• H3: height of Mount Everest
• H4: babies born per day in the United States

Each test is one-sided at α = .05, with no correction across items: each item is its own replication, as in Many Labs 1.

Design

• Materials. The four anchoring items from Many Labs 1, US version, word for word, including the instruction page (OSF project wx7ck, file Measures_US_Version.docx).
• Assignment. Every participant answers all four items in random order, one per screen. Each item's anchor (low or high) is drawn independently with probability .5 by the server when the session starts, as in the Many Labs 1 experiment definition (manylabs2.expt.xml).
• Response. An open text box labelled with the unit. The page asks for a readable number before moving on and never mentions a valid range.
• Debrief. After the fourth answer, participants see what the experiment was and the running results.

Sample and stopping rule

• Recruitment. Links shared on LinkedIn, Reddit (r/SampleSize) and elsewhere; a link may carry a source tag. Anyone can take part. There is no payment.
• Stopping rule. Collection stops at 200 analyzed sessions (after the session-level exclusions below) or on 2026-11-17, 45 days after launch, whichever comes first.
• Analysis set. The first 200 analyzed sessions in submission order, or all of them at the close date. Collection is never stopped or extended because of the results. The game may stay online afterwards, but later plays are not analyzed.
• Live results. The debrief shows running numbers during collection, labelled as running. They do not change the stopping rule.

Exclusions

Session level. These are added for an open online link (Many Labs 1 ran in labs and supervised panels):

1. The hidden form field that people cannot see was filled in (a bot).
2. Less than 8 seconds between starting and submitting, measured by the server.
3. A browser that already submitted, identified by a random token it stores: only its first session counts.

Response level. These are Many Labs 1's rules, unchanged:

4. An answer that cannot be read as a single number is missing. Separators, a trailing "+", hedges such as "about" and the item's unit are ignored; "thousand", "million", "k" and "m" scale the number. Ranges and other units are missing.
5. An answer below the low anchor or above the high anchor is missing. Both anchors bound every answer, whichever one the participant saw.

Analysis

Per item, following the Many Labs 1 confirmatory analysis:

• Rank-transform all valid estimates for the item (average ranks for ties), then run a one-sided Student t-test of the high-anchor group against the low-anchor group.
• Effect size: Cohen's d on the ranks (pooled standard deviation), with a 95% confidence interval from the large-sample standard error (Hedges and Olkin).
• Also reported: group medians and means, and the anchoring index: the high group's median minus the low group's median, divided by the distance between the two anchors (Jacowitz and Kahneman, 1995).
• Comparison with the original: whether our 95% confidence interval includes the Many Labs 1 weighted estimate (Table 2: d = 1.17 for distance, 1.79 for Chicago, 2.23 for Everest, 2.42 for births).

Robustness checks, reported but not used for the hypotheses: the same analysis without session-level exclusions 1–3, and Welch t-tests on the raw estimates.

Known differences from Many Labs 1

• A standalone task of about a minute, not 13 studies in one session.
• Online, self-selected and recruited through social media. Many Labs 1 had 36 samples: 27 in labs and 9 online.
• Participants see live aggregate results after they finish.
• The page requires a readable number before moving on.

Data and privacy

Stored per session:

• the four answers as typed;
• which anchor each question showed;
• the order of the questions and the time spent on each;
• start and submit times;
• an optional source tag;
• a random browser token;
• a salted hash of the IP address that changes every day, used only for rate limiting.

No names, emails or accounts. Aggregates and the anonymous estimates are published at /api/results.

References

• Jacowitz, K. E., & Kahneman, D. (1995). Measures of anchoring in estimation tasks. Personality and Social Psychology Bulletin, 21(11), 1161–1166.
• Klein, R. A., Ratliff, K. A., Vianello, M., et al. (2014). Investigating variation in replicability: A "many labs" replication project. Social Psychology, 45(3), 142–152. https://doi.org/10.1027/1864-9335/a000178

SHA-256 of PREREG.md: 620369bffd0c3d83d2a39a3fcf299b6afedb18393a055c09613e9ebdb7c40453