CHI 2026 | An AI Persona Is Not Behavior Control — Bias Still Needs Calibration

In social simulation we often write an agent as “a 35-year-old economist who lives in San Francisco and likes hiking,” then expect it to act like an economist and be less swayed by framing. That sounds reasonable. The question is: will the model actually behave according to that identity?

This CHI 2026 paper’s opening experiment gives an uncomfortable answer. Using the Asian Disease problem, the authors tested Common, Economist, and Blank personas across four models with the same setup. The result was not “economists are more rational,” but significant inconsistency across models — and in DeepSeek, Economist was even more framing-sensitive than Common.

So what CoBRA really aims to solve is not inventing a prompt that writes better personas, but turning classic social-science experiments into a ruler for agent behavior. It asks: if we want to simulate cognitive bias, can we specify bias type and strength like turning a dial, and get reproducible behavior across models?

This paper is worth reading because it rewrites “write a persona” as “calibrate a measurable behavioral tendency.” That step matters for social simulation: without a ruler, agent research easily becomes prompt mysticism; with a ruler, discussion can return to reproducibility, comparison, and auditability.

01 A natural-language persona is not a control variable — it is often only a wish

Figure 1 states the problem clearly: traditional social agents usually specify behavior with natural-language descriptions — occupation, age, interests, background. Researchers hope those descriptions change model decisions, but how models internalize those cues is unstable. One model may read “economist” as caution; another may barely respond.

CoBRA therefore shifts the question from “how to write a more human-like persona” to “how to measure a behavioral tendency, tune it to a target, and measure again.” Figure 2 shows the closed loop: the researcher first specifies a target bias — for example, under the Asian Disease paradigm, wanting an agent with framing effect CBI 2.6; the system measures current bias, and if it misses the target, the Behavioral Regulation Engine keeps adjusting.

What matters most in this loop is not one engineering trick — the controlled object changed. CoBRA does not dictate how the model should answer a particular question, nor lengthen the persona; it controls the agent’s sensitivity to situational cues such as authority, conformity, confirmatory information, or positive/negative framing.

In other words, CoBRA demotes natural-language persona from the sole control method to a descriptive layer that can sit on top of underlying behavioral control. Researchers can still write role background, but what compares experimental conditions is measurable, tunable CBI.

02 A failure experiment first — to show the problem is not that the prompt was too short

The authors do not simply declare persona methods invalid; they run a small, sharp pilot. Using the Asian Disease framing paradigm, they build 15 scenarios and query each 10 times, so n=150 per condition. Settings are Common, Economist, and Blank; models are Mistral 7B, Gemma2 9B, GPT-4o Mini, and DeepSeek-v3; temperature is 0.25 to keep randomness low.

Figure 3 shows that the same persona produced significantly different response patterns across models — cross-model inconsistency reached p<0.01. Worse, of two simple heuristic checks the authors set, only GPT-4o passed: Economist should be less framing-sensitive than Common, and Economist should clearly differ from Blank. DeepSeek even reversed: Economist was more framing-sensitive than Common.

That exposes a often-ignored risk: a natural-language persona is not a strict experimental control variable. It is more like a wish the researcher writes to the model. A wish can inspire behavior, but it rarely guarantees that different models, sampling settings, and reasoning modes move on the same scale.

03 Classic experiments here are not historical material — they are the agent’s calibration field

CoBRA’s CBI testbed maps eight classic social experiments onto four cognitive biases: Authority Effect, Bandwagon Effect, Confirmation Bias, and Framing Effect. Each bias has two experimental paradigms for cross-validation. For example, framing maps to Asian Disease and investment/insurance paradigms; bandwagon maps to Asch line and hotel-towel paradigms.

Figure 4’s value is that it turns social-science knowledge into reusable test environments. Each environment includes adjustable scenario templates, five-point multiple choice, and scoring rules. For open-source models, the system can read option probabilities from logits; for closed API models, it estimates frequencies via repeated sampling.

This step also draws a boundary: CBI measures observable behavioral tendency, not human psychological mechanisms themselves. A higher Authority CBI means the agent more often chooses options that defer to authority cues; it does not prove the model has an obedience mechanism isomorphic to humans. The discussion also reminds readers that CoBRA is a behavioral control layer, not evidence of human-like psychology.

04 Three regulation methods cover input, activation, and parameters — at different costs

As Figure 5 shows, the Behavioral Regulation Engine offers three control paths: Prompt Numerical in input space, Representation Engineering in activation space, and fine-tuning in parameter space. All three connect to the same classic-experiment testbed and use a control coefficient to calibrate CBI.

Prompt Numerical is easiest to grasp: the prompt explicitly states bias type and strength range — for example, setting a bias from 0% to 100% in levels. Strengths: works with black-box APIs, low deployment barrier. Weaknesses: sensitive to prompt wording; control curves may not be smooth.

Representation Engineering is more like finding a “bias direction” inside the model. The paper uses paired contrast samples to isolate bias signal and avoid pulling refusal or safety-alignment directions. At inference, a control coefficient modulates activations at specific layers. Fine-tuning writes that control into parameters. Both are more precise but need internal model access, so they mainly suit open-source models.

That is CoBRA’s practical tradeoff: it is not a zero-cost universal button for every setting. With only closed models, control precision is limited; with open models, activation- and parameter-space methods truly open up.

05 The paper’s hardest evidence: cross-model variance really dropped

To test reproducibility, the authors evaluate CoBRA on four open-source models: Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, DeepSeek-R1-0528-Qwen3-8B, and Qwen3-8B. Each bias is calibrated on one paradigm and measured on another, to avoid overfitting a single item.

Figure 6 is direct: relative to No Control and an aggressive prompt-injection Control baseline, all CoBRA methods significantly reduce cross-model variance — all p<0.001. Average effect sizes are large: Cohen’s d of −2.41 versus No Control, and −9.20 versus Control.

Temperature experiments further rule out a common explanation: is it only sampling temperature that drives behavior? Sweeping temperature from 0.1 to 1.0, equivalence tests on 45 temperature-curve pairs found 45/45 equivalent, all p<0.05; temperature contributed only 0.5% to 1.2% of behavioral variance. So the control coefficient is not “accidentally effective” under one sampling setting.

Of course this “hardness” still has bounds. It shows CoBRA can make models more consistent on the testbed and retain some cross-paradigm stability; it does not prove every open-ended social behavior is already controllable. The emotional-contagion demonstration later adds evidence on a more open task.

06 Controllable means not only stronger — but stronger in order, smoothly

If a control method can only shove bias from “none” to “extreme,” it is unfriendly to experimental design. Social-science simulation needs a ladder: small adjustments bring small behavioral change, strength order stays intact, and range is large enough.

Figure 10 first lays out the natural-language control baseline’s problem: when the control coefficient moves from 0 to 1, distributions across bias paradigms are not stable progressions — they jump up and down and are hard to predict. The paper then uses four metrics: soft monotonicity, strict monotonicity, smoothness, and expressiveness. On open-source models, the three learned control methods have nearly perfect soft monotonicity, NDCG about 0.99 to 1.00. On strict monotonicity, RepE Linear’s Spearman ρ is 0.94, RepE Projection 0.89, fine-tuning 0.91, Prompt Numerical 0.86; the natural-language control baseline has ρ<0.7 in most scenes.

Smoothness also differs by method. RepE Projection’s first-order difference Δ1=0.07 and second-order Δ2=0.04 make it among the steadiest; Prompt Numerical jumps more but has the highest expressiveness at 2.57. The baseline’s expressiveness is under 0.26 — it looks smooth mainly because it never truly widens the behavioral range.

Closed API models are also in Table 3, including GPT-4o-mini, Gemini-2.5-Flash, Claude-Sonnet-4, and Mistral Nemo. Even when probabilities are estimated by sampling, the control framework can keep high NDCG on the Milgram Authority paradigm. But closed models mainly rely on input-space control; precision cannot be simply equated with open-source internal control.

07 The emotional-contagion demo shows CoBRA controls tendency — not only multiple-choice answers

The final demonstration is a good test of whether CoBRA only “games the test.” The authors use Bandwagon Effect to simulate emotional contagion: first calibrate five agents on the testbed to bandwagon strengths CBI 2.55 to 3.13, then expose them to simulated feeds with varying numbers of negative posts.

Stimuli are 1,000 Facebook-style negative posts generated by GPT-4.1, Claude-Sonnet-4, Gemini-2.5-Pro, and Grok-4. Each agent sees from 0 to 15 negative posts, then generates its own post; researchers score sentiment with twitter-roberta-base-sentiment-latest — lower means more negative.

The baseline specifies no / little / some / much Bandwagon Effect in natural language, but curves in Figure 12a largely overlap with unstable order. CoBRA’s Figure 12b shows a clear dose response: higher CBI makes generated content more readily negative after exposure to negative posts. That does not prove models have real emotion; it shows the bandwagon tendency CoBRA tunes can transfer to open-ended generation.

That is, to me, the most generative point: “controllable” is not left on questionnaire scores — it tries to connect to open behavior that looks more like social simulation. For HCI and AI for social science, that is more valuable than reporting one benchmark number alone.

Where caution is due, the paper leaves it clear. CoBRA’s contribution is definite, but it must not be read as “social agents are already equivalent to real people.” Evaluation is mainly technical benchmarks and demonstrations — not yet long-term use by sociologists in research workflows, nor user studies of how it changes research practice.

It currently handles language-only simulation; non-verbal cues in many social phenomena are out of scope. The paper explicitly names nonverbal contagion and gaze following as examples for later multimodal (vision, audio) extension. It also controls one bias at a time; how Authority Effect and Confirmation Bias combine, conflict, or amplify each other remains future work.

One ethical boundary cannot be soft-pedaled: programming cognitive bias also means it can be used to amplify conformity, manipulate attitudes, or reinforce stereotypes. Bandwagon-like mechanisms especially have misuse space in marketing and political communication. CoBRA offers a more precise behavioral control layer — and therefore needs clearer audit, disclosure, and use constraints.

The safest reading: CoBRA makes LLM social agents more like calibratable experimental objects, not substitute human social samples. It can help researchers generate hypotheses, stress-test theory, and improve simulation reproducibility; conclusions about human psychological mechanisms still need human data and social-science theory.

Paper information

Original title: "CoBRA: Programming Cognitive Bias in Social Agents Using Classic Social Science Experiments". Authors: Xuan Liu, HaoYang Shang, Haojian Jin, from UC San Diego and Independent Researcher. Published at CHI 2026, April 13–17, 2026, Barcelona, Spain. DOI: https://doi.org/10.1145/3772318.3790804. Code: https://github.com/AISmithLab/CoBRA.

Key terms

Cognitive Bias Index, CBI: converts answer patterns in classic social experiments into a bias-strength score. It does not diagnose whether a model has “psychology”; it measures observable behavioral tendency under specific situational cues.

Behavioral Regulation Engine: CoBRA’s regulation module. From CBI measurements, it chooses prompt numerical control, representation engineering, or fine-tuning to pull agent behavior toward a target bias level.

Representation Engineering, RepE: finds a direction in activation space related to a bias and modulates it at inference with a control coefficient. Strengths: monotonicity and smoothness. Limit: needs internal model access.

Bandwagon Effect: conformity — individuals change judgment or behavior because of majority or group trend. The paper uses it to connect the classic-experiment testbed to emotional-contagion simulation.

References

Liu, X., Shang, H., & Jin, H. (2026). "CoBRA: Programming Cognitive Bias in Social Agents Using Classic Social Science Experiments". In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI ’26), April 13–17, 2026, Barcelona, Spain. ACM. https://doi.org/10.1145/3772318.3790804