Short answer: A chatbot can run short, text-based qualitative interviews cheaply and at scale, but not yet deep ones. In a 2026 study, a Gemini-powered chatbot interviewed 74 social scientists for a total model cost of €0.04 and produced 149 pages of data, yet the authors concluded it did not reach the depth and nuance a qualitative interview usually aims for.
AI chatbot interviews promise something qualitative researchers have rarely had: dozens of adaptive, semi-structured conversations running at the same time, with no scheduling and almost no marginal cost. A new open-access article by Tullia Jack, Alex Cooper and Lisa Flower in Methodological Innovations (2026) tests that promise on an unusually demanding audience: professional social scientists who know what a good interview looks like. Their verdict is useful precisely because it is mixed.
\n\nThe study asked whether an LLM can work as a qualitative interviewer, using a chatbot to interview researchers about their own use of generative AI. The paper is Automating the qualitative interview? Using Gen AI chatbots in social science research (Methodological Innovations, Vol. 19, Issue 3, first published May 27, 2026).
\n\nThe authors built their own web chatbot on Google's Gemini 1.5 Flash, accessed through the API, and prompted it to follow a semi-structured topic guide with four themes: current use of generative AI, existing coding practices, ideal future uses of AI in analysis, and ethics. The prompt told the model to open each theme with a broad question, probe with follow-ups, avoid sensitive topics, and move on if someone declined to answer. Participants were told the chat would take around 10 minutes.
\n\nThat gives 74 chatbot interviews plus 23 follow-up emails. Humans, not the model, did the main analysis.
\n\nOn efficiency, it performed very well. According to Jack, Cooper and Flower (2026), the whole project cost €0.04 in model fees (excluding researcher salaries), most conversations lasted 8 to 12 minutes, and more than four-fifths of participants finished the interview. Completed chats ranged from 1 minute 49 seconds to 33 minutes 20 seconds.
\n\nThe bot also showed real interviewing skill. It summarised points back for confirmation, suggested ways forward when someone seemed stuck, recognised field-specific acronyms, and followed participants into topics outside the guide. The authors saw no hallucinations in the chats, and it stayed away from sensitive topics as instructed. In a footnote, the first author writes that she was humbled by how well the bot got people talking.
\n\nThe failures were mostly about conversational pacing and leading. The authors document several patterns that a skilled human interviewer would avoid:
\n\nOf the 23 follow-up emails, 11 were broadly positive, 8 were neutral or mixed, and 4 described feeling disengaged or mildly annoyed. Some participants said talking to a bot made it easier to be blunt and honest.
\n\nThe study's findings line up as a trade-off between scale and depth.
\n\n| Dimension | What the chatbot achieved | What it lost |
|---|---|---|
| Cost and logistics | €0.04 in model fees for 74 interviews; simultaneous, no scheduling | Needed custom web development, API work and prompt iteration |
| Volume | 149 pages of text, a little under half of it from participants | The authors stress that quantity is not the measure of a good interview |
| Probing | Summarised, clarified and handled expert vocabulary | Missed probes, stacked questions, cut people off |
| Neutrality | Perceived as non-judgmental; some answered more candidly | Introduced at least one leading assumption |
| Attention | Participants could answer whenever suited them | Signs of multitasking, such as one sporadic 51-minute chat |
The authors' bottom line is direct: chatbots may suit some research questions, perhaps as a complement to questionnaires, but are currently not appropriate for gathering thick data.
\n\nOnly as a starting point. The authors gave all 74 transcripts to Gemini and asked for Weberian ideal types of how social scientists relate to AI. The model proposed five types; the researchers merged overlapping ones into three (Enthusiasts, Pragmatic Adopters and Gen AI Agnostics). The quotes the model cited were accurate about 80% of the time, and it claimed younger researchers were more enthusiastic, a pattern the authors could not find in the data. They also note how tempting it was to keep re-prompting rather than stop and reflect.
\n\nThe participants were in a similar place. Of the 74, 37 were experimenting with generative AI, mostly for writing support, translation, transcription and literature searches. Few used it for analysis, and many worried about black-box coding, reproducibility and data security.
\n\nThe study works as a checklist of failure modes to test for before fielding an AI-moderated study:
\n\nMost of these can be caught in a pilot. Our guide to pilot-testing an AI interview guide covers how to run one. If you use a hosted tool such as the Qualitati AI Interviewer rather than building your own bot, the same tests apply: read pilot transcripts specifically for interruptions, stacked questions and leading framing before you scale up.
\n\nTwo limits matter when reading the results. The sample was small and self-selected, and the authors expect many participants were already interested in AI. The bot also ran on Gemini 1.5 Flash, and newer models may behave differently.
\n\nModel costs can be tiny. In Jack, Cooper and Flower (2026), 74 text interviews cost €0.04 in API fees. That excludes development, prompt design and researcher time, which were the real costs.
\n\nNot for in-depth work, according to this study. The chatbot gathered a lot of data and engaged participants, but the authors judged that it did not reach the depth and nuance of a good qualitative interview.
\n\nSome participants said so. Several described the bot as non-judgmental and easier to be blunt with, though others felt disengaged or found it odd that an AI was collecting the data.
\n\nGoogle's Gemini 1.5 Flash, accessed through the API inside a custom Python and JavaScript web application.
\n\nLast updated: October 6, 2026. This is an independent editorial summary of third-party research published in Methodological Innovations; Qualitati is not affiliated with the authors. See the original article for full methods and findings.
","related":[{"slug":"synthetic-survey-panels-marginal-fidelity-2026","title":"Do Synthetic Survey Panels Simulate People? A 2026 Test","description":"Do synthetic survey panels simulate real respondents? A 2026 test finds matching published averages is not proof: a no-persona prompt beat panels 9 of 9 times.","category":"Methodology","date":"2026-10-22","readTime":"7 min read","author":"Qualitati Research Team"},{"slug":"llm-as-judge-qualitative-interpretation-2026","title":"Can LLM-as-Judge Rate Qualitative Interpretations? A 2026 Test","description":"Can LLM-as-judge replace human review of qualitative interpretations? A 2026 study of 3,560 AI interview readings finds it screens models but misses nuance.","category":"Methodology","date":"2026-10-21","readTime":"7 min read","author":"Qualitati Research Team"},{"slug":"rehearse-interview-protocol-simulated-participants-2026","title":"Can You Rehearse Interviews with AI Participants? A 2026 Study","description":"Can you rehearse an interview guide with AI-simulated participants? A 2026 study of 21 researchers found real protocol revisions and three hidden risks.","category":"Methodology","date":"2026-10-21","readTime":"7 min read","author":"Qualitati Research Team"}]}