esc
All posts

Let the model vote, not summarise

5 min readWind Tunnel

On this page

“Starting next month, everyone is back in the office five days a week.” Ask a chatbot whether that is safe to announce and you get one answer, and the question ends there. Wind Tunnel shows the same sentence to a hundred synthetic personas and returns a map of positions instead of an answer: four of them, each with the voices of real reactions. The question is still open when you finish reading, and the judgement stays with you.

One step makes the map possible: turning a hundred free-text reactions into a vote matrix, with the model as the voter. This post is about that step only.

The Wind Tunnel example on the projects page: the announcement, then one card per position, each with its name and a voice from it

Summarising is not enough

Here is one of the hundred, from a 45-year-old shipping clerk in North Carolina:

Warehouse crew never left, so welcome back to reality, but forcing five days in a cubicle just screams insecure management who don’t know how to measure actual output. Good luck with the turnover.

Three claims in one breath. From the shop floor, remote workers’ complaints look soft. A mandate is what management does when it cannot measure output. People will leave.

Ask a model to summarise the hundred and you get a paragraph that restates the majority. An earlier version of Wind Tunnel let a model name each position, and the names came out as paraphrases of the majority view; the differences between positions vanished. Embed the voices and cluster the vectors, and they group by how they are written. Give each voice one label, and the clerk’s second and third claims are lost. To sort by claims, the claims have to come first, and everyone has to vote on them. It is what Pol.is does with real crowds, done with synthetic voters.

Extracting the propositions

A strong model (Gemini 3.1 Pro) reads all hundred voices and extracts ten to fifteen propositions that a person could agree or disagree with. Three constraints: each must be specific, each must actually have been stated by somebody, and they must not overlap. The output is [{id, text}]; if it fails the schema, the positions analysis fails rather than guessing. The office run produced twelve, among them:

  • p1: A blanket five-day return-to-office mandate indicates that management does not know how to measure actual employee output.
  • p2: Forcing employees back to the office five days a week will lead to a significant increase in staff turnover.
  • p3: Remote workers complaining about returning to the office lack perspective on the reality of essential and trades workers who never had the option to work from home.

The propositions set the map’s resolution. One too broad draws agreement from everyone and makes no axis; two that overlap count the same axis twice.

Letting the model vote

A fast model (Gemini 3.8 Flash) reads the voices ten at a time and returns, for each voice and each proposition, +1, −1 or 0, as an array of objects keyed by proposition id. Twelve batches run in parallel, so a hundred voices are ten batches and one round. The result is a 100 × 12 matrix. The clerk above reads as agreeing with p1, p2 and p3 at once, and the matrix holds that as one row.

How failures are handled decides the quality of the map. A batch that returns broken JSON, or an element with no proposition id in it, becomes null, and that voice is left out of the map. An earlier version filled it with zeros, “neither”, and every bad minute the model had grew a neutral blob in the middle of the map. A failure must not look like a position. If fewer than half the voices get their votes, there is no map at all.

The classifier is also told not to think. Gemini 3 models reason before they answer, the reasoning is billed as output tokens, and a call’s time follows its output. Classifying ten voices against twelve propositions is a short answer, yet batches took anywhere from 5 to 71 seconds, the slowest batch held everything behind it, and a production run used its whole 280-second budget. With the thinking level set to low, a batch takes 4 to 6 seconds and emits under 900 tokens instead of up to 12,056, and the map and the issues read at the same level.1

With the matrix, the rest is arithmetic

No model is needed from here. k-means runs on the rows of votes, with k chosen by silhouette score. Each position’s representatives are the three members nearest its centroid, so the voice on each card is a real reaction. A proposition is consensus only when every position agrees with it at a rate of at least 0.6. The minority is the smallest cluster, reported through the propositions on which it differs most from the weighted average of everyone else. A model writes prose only at the end: a name and a one-sentence belief for each position.

No counts are shown. The proportions among synthetic personas measure nobody. What is shown is each position, its voices, and a meter for where every position stands on the propositions that set it apart. Answer a few of the propositions yourself and your own point appears on the map; the positions far from it are candidates for your blind spots.

The detail of one position: its core belief, two propositions with a meter showing where this position and the others stand, and three of its voices with age, occupation and town

Limits

  • The propositions come from the same voices they sort. A position nobody in the hundred took is not on the map.
  • The votes are a model’s reading, and a failed batch takes its voices with it.
  • Each persona speaks once, alone. Nobody replies, and nothing escalates.

Try it

The demo runs on your own text. The CLI (Apache License 2.0) runs the same steps on your machine with Ollama and leaves the vote matrix as CSV. With the matrix in hand, the arithmetic in this post can be redone by hand.

Terminal window
npx wind-tunnel-cli doctor
npx wind-tunnel-cli personas pull usa
npx wind-tunnel-cli run "Starting next month, everyone is back in the office five days a week."