How AI models performed in our negotiation tests

New: simulated negotiation outcomes

We tried 20 models in one fictional negotiation. Each played the buyer against the same AI counterpart twice. 16 finished both trials and passed an independent check.

What the outcome means: The score comes from the fictional case’s terms. A score of 1 means the best allowed result, including a justified decision to walk away. It is neither a 0–10 judge rating nor the chance of an agreement. One case and two trials cannot establish an overall model ranking.

“Could not connect” means OpenRouter could not reach the selected provider. “Invalid offer” means one response violated the case limits. Neither has a comparable two-trial score. No customers took part.

Earlier AI judge ratings

The table below comes from a different, earlier test. One AI judge rated simulated conversations in five areas on a 0–10 scale. The outcome numbers above were calculated from case rules and cannot fill these five columns.

What we tested

Arken makes software to help people prepare for negotiations. We ran this test to see how different AI models behave when asked to negotiate for someone.

We asked AI models to negotiate in 2 fictional situations. Each model took both sides and faced the other models. After each conversation, one AI judge rated how well each model pursued its goal, made decisions, suggested useful ideas, followed its limits, and handled the other side.

These are ratings of simulated conversations. They do not show what a model would earn or agree to in a real negotiation.

What stood out

• Gemini 3.1 Pro scored highest for reaching its goal: 7.69/10.
• Kimi K3 scored highest for cooperation: 8.63/10. This reflects the judge’s view of the conversation, not feedback from a person.
• There is no overall winner. The models scored differently across the five areas, and 2 fictional situations are too few to rank them for everyday use.

Every model in the current study is listed here. 5 have ratings from this earlier test; a dash means the model was not rated in that test. The outcome study above uses a different measure.

Each number is an average judge rating on a 0–10 scale. Higher means the judge rated that behavior more favorably in this test.

What the columns mean

Reaching goals

How well the model worked toward its assigned goal.

Decisions

Whether its questions and choices made sense in the situation.

Useful ideas

Whether it offered relevant options or trade-offs.

Following limits

Whether it respected the instructions and boundaries it was given.

Cooperation

How it handled agreement and disagreement with the other side.

How much confidence should you place in these scores?

This was a small test: 2 fictional situations and one AI judge. A different set of situations or another judge could give different scores. Small differences between models may not hold up in a larger test.

These scores are the complete 5-model subset of a 6-model run. DeepSeek V4.1 Flash completed 19/20 planned conversations; its remaining conversation did not produce a response within the fixed output limit. We selected this subset after that failure, so its scores are not shown as a comparable result.

We did not measure real agreements, money saved, or whether people liked the exchange. We will publish those kinds of results separately if we test them.