The first Turing Text Edition is in progress as we speak. Based on a typical romance fraud scenario, our participants join the fictional dating app, La Vida Lanca, and have a chat with Alex. The premise, as in the rest of the suite of quizzes, is that participants try to work out whether they are interacting with a bot… or not. There are five simple steps:
- Just like the other perception tests, participants firstly rate how confident they are in their ability to detect who – or what – they will be speaking to.
- They ask Alex a question, Alex answers and then asks a question in return, and on it goes until each has asked and answered three questions apiece.
- Each side then privately tells the moderator whether they think they’ve been chatting with a human or an AI, and what giveaways or clues led them to that conclusion.
- Each side also gives their confidence in this answer. After all, the interaction may have made them much more or less sure than they were to begin with.
- And finally, the moderator reveals the truth and tells them if their partner was a bot, or not.
As you’d expect, the “dating” chat is randomly assigned. It could be two humans or it could be a human and an AI, but we’re also running these trials on two AIs. Yep. After all, what if bots spot bots better than humans? Or, interestingly, what if they’re worse? How well do different generative AI models perform at pretending to be a human on a dating app? Will ChatGPT be more persuasive – and perceptive – than Gemini? Is Claude more alluring and astute that Grok? What of Copilot? Deepseek? Llama? Qwen? Mistral?
As all this suggests, our pilot is still in progress, and we expect results to start arriving by late 2026.
Acknowledgements
Our Turing Text Edition pilot is only possible due to the support of Security Lancaster, and the tireless work of our RA, Hope McVean.