Short answer: No, not reliably. In a 2026 study of 12 real first-click tests with 3,431 human participants, GPT-simulated clicks followed a significantly different distribution from real users in 53% of tasks. Personas, chain-of-thought reasoning and sampling settings made the synthetic data look more believable, not more accurate.
Synthetic participants are pitched as a cheap, instant stand-in for user testing. A new preprint from researchers at the Slovak University of Technology and UXtweak puts that claim to a practical test: can a large language model predict where people click on a real interface, and explain why, well enough to steer design decisions? According to Kuric, Demcak and Krajcovic (2026), the answer is a clear no, and the way it fails matters for anyone tempted to replace a quick usability test with an AI simulation.
What did the study test?
The study compared GPT-simulated first-click tests with the real results of the same tests. First-click testing shows participants a static screenshot or wireframe, gives them a task, and records where they would click first. The authors chose it because it is one of the simplest and most controlled UX methods: if a model cannot simulate a single click on a single image, it is unlikely to simulate a multi-step user flow.
The dataset came from real practice, not a lab. The authors obtained 12 first-click studies run by different organizations on the UXtweak platform, covering 3,431 participants, 45 tasks and 120 follow-up questions. Domains included healthcare, public administration, social services, entertainment and transportation; nine studies tested desktop websites, two tested mobile apps and one tested diagrams.
Each study was re-enacted with GPT-4.1 as the baseline, using the same instructions and images the human participants saw, plus a persona built from the real sample's pre-study questionnaire answers. The team then varied one factor at a time, across 84 simulations in total:
- A newer reasoning model: GPT-5.2 with reasoning effort set to none and to medium.
- Lower temperature: 0.2 instead of 1.
- Narrower nucleus sampling: top_p of 0.2 instead of 1.
- Individual personas (one simulated person at a time) instead of a single population-level "mega-persona".
- Generic personas instead of personas matched to the real sample.
How accurately did GPT predict first clicks?
GPT often guessed the most popular target but badly misjudged everything else. According to Kuric et al. (2026), the baseline model's click distribution differed significantly from real users in 53% of tasks. Its top choice matched the human top choice in 71% of tasks, but rank-order agreement across options averaged only .16, and the mean Jensen-Shannon distance was .48 on a 0–1 scale.
The synthetic data was also far too uniform. Real participants clicked an average of 9.00 distinct hotspots per task; the simulation spread its clicks over 5.38. In practice, GPT tended to crown one "victor" element that shared words with the task and pile most clicks onto it, whether or not it was the right answer. It also added spurious clicks on elements real users ignored and omitted options that real users clicked often. The authors note that these marginal clicks create an illusion of variability.
Did synthetic follow-up answers match what real users said?
Only partially, and the open-ended answers were the weakest part. For closed questions such as Likert ratings, the answer distribution differed significantly in 45% of questions, and the top answer matched in just 50%. Synthetic Likert answers clustered around the highest or second-highest rating regardless of the question, while real data showed neutral, negative and multimodal patterns.
Open-ended answers were semantically close to human text on the surface (mean semantic similarity .70) but significantly less diverse and less readable, and they were shorter: 9.08 words on average versus 13.31 for humans. The authors' reflexive thematic analysis of these answers identified five recurring distortions:
- Superficiality and homogeneity: generic praise or complaints with no experiential reason behind them.
- Hallucinations: comments on loading speeds, icons or content that did not exist in a static image.
- Positivity and helpfulness distortion: mostly positive, hedged criticism, and an answer to every optional question even when real people had nothing to add.
- Literal interpretation and over-roleplay: treating a screenshot as a product used in real life, or answering "anything else to share?" with a bare "no".
- Spurious heuristics: calling a page "family-friendly" because the word "family" appeared in the task scenario.
Do personas, reasoning or temperature fix synthetic participants?
No setting closed the gap, and some made it worse. The table summarizes what Kuric et al. (2026) reported for each lever.
| Lever tested | Effect on fidelity to real users |
|---|---|
| GPT-5.2 with reasoning (medium) | Slightly more varied clicks (entropy .54 vs .43 baseline, still below real data); closed-question alignment worse; open-ended diversity lower |
| Temperature 0.2 | No significant effect on clicks or closed questions; slightly less diverse open-ended text |
| top_p 0.2 | No significant effect on clicks or closed questions; less diverse open-ended text |
| Individual personas | Much worse: significant distribution difference in 96% of tasks, with almost every persona clicking the same single option |
| Generic vs sample-matched personas | No significant difference in clicks or open-ended text |
The persona finding is the most practical one. Simulating many individually described users did not produce a diverse sample: the average number of unique options chosen dropped to 1.16. Detailed demographic and Big Five profiles changed how answers read, not where the model clicked. The authors trace these failures to how LLMs work: they predict likely text from language patterns rather than perceiving an interface and acting on it.
What this means for UX and market researchers
Treat synthetic first-click or usability data as a hypothesis generator at best, never as evidence for a design decision. The study's pattern of a confident single "winner", missing secondary paths and upbeat feedback is exactly the kind of output that looks convincing in a stakeholder deck while hiding the confusion real users would show.
Practical takeaways from the findings:
- Do not use simulated clicks to choose between designs. A 71% top-choice match still means the simulation pointed to the wrong winner in roughly three tasks out of ten, and it under-reports alternative paths.
- Be skeptical of persona-rich synthetic panels. More persona detail did not make the simulated sample behave more like the real one.
- Watch the tone of synthetic feedback. Uniformly positive, tidy comments are a warning sign rather than good news.
- Keep real participants in the loop and make them cheaper to reach instead. Short asynchronous sessions with an AI interviewer collect real users' reasoning at scale. If you do use simulated respondents, validate them against real data first, for example with the Digital Twin Panel, rather than trusting them by default.
The authors acknowledge limits: only OpenAI models were tested, single runs were compared after pretests showed little run-to-run variation, and some original tasks used wording that hinted at the answer. Even so, first-click tests are among the easiest UX tasks to simulate, which makes the results hard to dismiss.
Frequently asked questions
Can AI replace real participants in usability testing?
Not based on this evidence. In 12 real first-click studies, GPT's simulated clicks differed significantly from human behavior in 53% of tasks, and its follow-up answers were more positive, more generic and less varied than real feedback.
Does a better reasoning model make synthetic users more accurate?
Not meaningfully. GPT-5.2 with chain-of-thought reasoning spread its clicks slightly more but did not fix the core mismatches, and it aligned worse with humans on closed follow-up questions than GPT-4.1.
Do detailed personas improve synthetic participant data?
In this study, no. Personas matched to the real sample performed no better than generic ones, and simulating users one persona at a time made the results far less realistic.
Where can I read the original paper?
The preprint is Kuric, Demcak and Krajcovic (2026), "What Would GPT Click: Practical Effects of Human-AI Behavioral Misalignment and the Cost of Synthetic Participants in User Experience", arXiv:2605.18302 [cs.HC].
Last updated: October 7, 2026
This article is an independent editorial summary of third-party research by the Qualitati Research Team. It is not affiliated with or endorsed by the paper's authors. Figures are reported as published in the arXiv preprint; consult the original for full methods and results.