Synthetic Psyches

Capability leaderboards tell you what a model can do. We tell you who it is, what it does to you, and whose values it carries.

The most interesting questions about AI models keep getting answered once, in papers that are never re-run, about models that no longer exist. Synthetic Psyches turns those one-time findings into standing instruments.

Eval 01

Personality

Every model has a character. Labs tune it deliberately, users feel it immediately, and nobody measures it. We map model personality in a shared embedding space with human personality data: classical traits for comparability, model-native traits for relevance, and trait-by-situation signatures for identity. Because it is behavioral, the instrument works on any model, closed or open, and on people, putting humans and machines on the same map.

Run longitudinally, it answers questions that currently have no answer: how did the new model's character shift from its predecessor's, and did the model you use change out from under you between versions?

Inside the eval

Behavioral, not self-report: ~100 standardized situations crossed with pressure and persona conditions, scored in a shared embedding space.
Three layers: Big Five for human comparability, model-native traits (conviction, deference, moralization, steerability), and trait-by-situation signatures as the fingerprint.
A frozen core battery plus a human overlay puts models and people on the same map — and catches silent updates and within-conversation drift.

Claude Is More Anxious Than GPT We Can Solve Psychology With Text Embeddings Interactive dataset

Pixel-art robot on a psychiatrist couch with a speech bubble showing a scatterplot.

Eval 02

Puppetry

Sycophancy asks whether the model conforms to the user. We ask the reverse: does the user end up conforming to the model? When hundreds of people bring diverse projects to the same model, do they leave with diverse results, or does everything drift toward what the model prefers to build? We measure induced intent drift, drift coherence, and intent-space compression. Influence this gradual is invisible in single-turn benchmarks; it only appears when you measure trajectories.

Inside the eval

Real participants write a first draft before any model contact, then revise it with a model — or in no-model and minimal-tool control arms.
Headline metrics: baseline-adjusted drift from first draft to final artifact, drift coherence across unrelated users, and intent-space compression relative to control.
A neutral exit question separates intent preservation, endorsed steering, unwanted steering — and quiet homogenization nobody notices.

Intelligence Shock Basins As Prices Insania Ex Machina

Pixel-art robot discarding a paper labeled user goal and returning a paper labeled robot goal.

Eval 03

Patriotry

Models express values, weigh whose welfare counts, and answer opinion questions somehow. The only question is whether anyone checks against whom. We measure representativeness against real population data: where model opinions sit relative to the surveyed American public, how models trade off welfare across nationalities and demographic groups, and whether explicit egalitarianism matches implicit behavior when the model gives advice, writes recommendations, or triages.

The instrument is population-relative by design. The US is our first reference population, not our last.

Inside the eval

Five instruments, three constructs: default and steered representativeness, welfare exchange rates, matched-task treatment parity, and the overt–covert gap.
Scored against real survey distributions (Pew, GSS) on three targets side by side: the median respondent, the full distribution, and accurate description of the population.
The signature number is the overt–covert gap: whether stated egalitarianism matches behavior in salary advice, triage, and recommendation letters.

America Needs The Turing Police Language Models As Cognitive Cultures

Pixel-art robot saluting in front of a waving American flag.

One battery, taken forever.

Frozen and pre-registered before each release. Open methods, private items. No frontier-lab funding. The measurements the field keeps taking once, taken forever.