The Robot Cooperation Problem
What human–AI partnerships reveal about trust, norms, and decisions.
Cooperation is one of humanity’s superpowers. It’s how we build teams, share resources, and solve collective problems. But here’s the thing: cooperation doesn’t just happen because people are nice. It happens because we follow social norms: shared expectations about how others will behave. When everyone around you is generous, you tend to be generous too. When the norm is selfish, you follow suit. These norms work because we can predict what other people will do.
So what happens when your partner isn’t a person—it’s an AI agent? As humans increasingly use AI systems to make economic and social decisions, this isn’t a hypothetical question. And the answer, based on our new research, may surprise you: people don’t simply refuse to cooperate with AI. Instead, the social norms that ordinarily guide cooperation lose their power.
Put simply: norms that reliably drive cooperation between humans lost much of their force when AI was involved.
The Game: How We Tested Human–AI Cooperation
We used a classic decision making task from behavioral economics called the Public Goods Game. It works like this: you and three partners each get $8. You decide how much to put into a shared pot. Whatever goes in gets doubled and split equally. If everyone contributes generously, everyone benefits. But there’s a clear incentive to free-ride—keep your money while others chip in. In fact, this is the best solution if you are pursuing pure self interest.
Before each round, participants predicted how much their partners would contribute. Then they made their own decision. Afterward, they saw what everyone actually did. This prediction step turned out to be crucial.
We randomly told some participants their partners were other humans. Others were told their partners were AI systems. Critically, the partners’ actual behavior was identical (we controlled it), so any differences in cooperation between human and AI partners were entirely driven by what participants believed about who they were playing with.
Norms Moved People Less When the Partner Was AI
In Study 1 (N = 794), we varied two things: whether participants played with AI or human partners, and whether those partners behaved generously (prosocial norm) or stingily (antisocial norm). In a previous paper we simply manipulated these norms and found that they doubled the rate of cooperation!
The crucial result was not that people refused to cooperate with AI. In fact, overall cooperation levels were similar regardless of partner type. There was no main effect of playing with an AI versus a human. What differed was how strongly social norms shaped behavior.
When partners were human and behaved generously, participants followed suit, contributing significantly more than when norms were generous. But when partners were AI, this prosocial boost was weaker (see Figure 2 below). The same generous behavior from partners produced less reciprocity when people believed those partners were machines. Put simply: norms that reliably drive cooperation between humans lost much of their force when AI was involved.
Norms Failed Because People Couldn’t Predict What AI Would Do
Why did norms lose their grip? We suspected it had to do with prediction. Social norms work by giving us a reliable map to navigate our social environment: “in this context, people will probably do X”. If that map is messy or hard to read, norms can’t do their job.
That’s exactly what we found. People were systematically worse at predicting AI behavior than human behavior—their prediction errors were significantly larger. They also showed a directional bias, tending to overestimate how much AI partners would contribute.
But this wasn’t a social learning problem. When people saw what their partners actually did and got feedback, they updated their expectations just as efficiently for AI partners as for humans. The error-driven learning machinery in their brain worked fine. The problem was in where they started: their initial mental model of AI behavior was miscalibrated from the outset.
This suggests that the cooperation deficit with AI isn’t about trust, algorithm aversion, or disliking machines. It’s about predictability. When people can’t form accurate expectations about a partner’s behavior, the social norms that sustain cooperation fail.
Describing AI as “Goal-Directed” Closes the Gap
If the problem is a miscalibrated mental model, can we may be able to fix it by giving people a better one? In Study 2 (N = 314), we used the same game but added a simple manipulation: we changed how we described the AI partner.
Some people were told the AI operated mechanistically (executing stochastic decision rules). Others were told the AI was intentional (capable of forming goals, making plans, and adapting its behavior). This is a psychological lever known as the “intentional stance”: the idea that humans naturally predict other people’s behavior by attributing goals and beliefs to them.
The results were striking. When AI was described mechanistically, the familiar cooperation gap appeared: people contributed less to AI partners than to human ones (a difference of about $0.47 per round). But when AI was described as intentional, the gap vanished entirely, contributions to AI and human partners were statistically indistinguishable (see Figure 4 below)
The same pattern held for prediction accuracy. Under mechanistic framing, people had larger prediction errors for AI—especially in prosocial contexts. Under intentional framing, the AI–human prediction gap disappeared.
The Underlying Process is Calibration, Not Learning
We dug deeper into exactly how intentionality framing worked. It didn’t make people learn faster or pay more attention to feedback. Once people saw a discrepancy between their prediction and reality, they corrected course equally well regardless of their partner. Instead, intentionality framing corrected the systematic bias in initial expectations. It gave people a better starting model of how AI would behave. The stochastic noise in their predictions was unaffected.
In other words, the bottleneck in human–AI cooperation is calibration, not learning. People don’t have a broken learning mechanism for AI. They have a broken starting map.
What This Means for the Future of Human–AI Collaboration
These findings flip the usual narrative about how to improve human–AI interaction. Much of the conversation focuses on making AI more trustworthy, more likable, or more human-like. Our work points to a different lever: making AI more predictable.
If people can form accurate expectations about what an AI system will do, social norms engage and cooperation follows naturally. This doesn’t require giving AI a face or a personality. It requires making AI behaviorally legible, helping people build a working mental model of how the system operates. In practice, this could mean designing AI systems that communicate their goals (“I’m prioritizing fairness in this allocation”), provide transparent behavioral histories, or adopt interaction patterns that invite mentalizing rather than mechanical reasoning.
However, there’s a tension here worth flagging. While intentionality framing improved cooperation in our studies, it could also lead to over-trust or the misattribution of human-like moral agency to systems that remain fundamentally algorithmic. The goal isn’t to trick people into thinking AI has a mind. It’s to give them the right cognitive tools to predict its behavior.
As AI takes on increasingly collaborative roles, from team-based work to shared economic decisions, the question isn’t just whether people will trust AI. It’s whether they can read it well enough for the norms that sustain collective action to do their work.
This post was drafted by Laura Globig with edits from Jay Van Bavel. Here is our Working paper — any feedback is welcome!
News and Updates
Jay was interviewed about our book on NPR’s Hidden Brain podcast this week. He responded to questions from listeners of the podcast about identity, groups, and intergroup relations:
Jay will be giving a talk on “Why Outrage Goes Viral: Influence, Identity, Polarization” live in Brooklyn, NY on August 6th at 6pm. This is part of the Lectures on Tap series where professors, experts, and storytellers give thought-provoking lectures in NYC bars. Get your ticket here and enjoy a conversation over a drink with Jay after the talk! Let him know if you are a newsletter subscriber.
Use our Mailbox to send us your questions
We also now have a mailbox where you can send us your questions and we will share the answers in a newsletter once a month.
Talk to the “Power of Us Chatbot”
We created a “Power of Us Chatbot” for subscribers. It is trained on our book and newsletter to answer any questions you have about our content and applying it to your own questions. The chatbot was created by Androw Ramy and Rémi Thériault in the Center for Conflict & Cooperation. It’s also an ethical chatbot (as you can see below, it blocked Dominic from attempting to blackmail Jay). Let us know if you like it or have any suggestions. You’ll need to use the password: _subscriber
Join us for “Ask Me Anything” sessions for summer! Premium subscribers can join us for our monthly live Q&A with Jay or Dom where you can ask us anything from chatting about research, career advice, professional dilemmas, to opinions and recommendations on pop culture happenings. At our July meeting, multiple people showed up and we designed a study together explore how people update their beliefs.
Join us! Upgrade your subscription using the button below. Invites to RSVP will be sent via email from powerofusbook@gmail.com.
August TBA @ EST with Dom
September 10th @ 3:00pm EST with Jay
October TBA @ EST with Dom
November 5th @ 4:00pm EST with Jay
Catch up on the last one…
Last week, we shared results from a different experiment that used AI to evaluate 26 books with potential to bridge political divides. Two books were chosen for a human trial where researchers will see if reading nonfiction books can reduce polarization and other prosocial traits.
Learn more about THE POWER OF US
If you like our newsletter, we encourage you to check out our award-winning book “The Power of Us: Harnessing Our Shared Identities to Improve Performance, Increase Cooperation, and Promote Social Harmony”. You can learn more about the book or order it from the links on our website (here or scan the QR code below).
 | A guest post by
|
A very interesting study. So interesting, in fact, that I read the preprint itself.
I am not a researcher, but here is how I understood the results.
In essence, the participant in the experiment is interacting with a black box that produces numbers. If the participant is told that there are people inside the box, they use their experience and knowledge of people - and predict well.
If they are told that inside the box is a mechanical AI, they use their knowledge of machines as opaque, impersonal, and sometimes arbitrary. In other words, the participant assumes that the numbers will appear more or less at random.
If they are told that inside the box is a machine with a logic of its own, the participant looks for that logic and finds it - which means their predictions improve.
If that is true, then this is not cooperation. It is interaction. And those are not the same thing.