Welcome to the Bot or Not? project – a suite of perception tests designed to find out some simple yet critically important questions important:
How good are humans (and AIs) at detecting AI-generated content versus human-created content?
Are humans (and AIs) under- or over-confident at this task?
What do humans (and AIs) focus on when trying to tell the difference, and are they focussing on the right things?
Simple beginnings
Our interest in this all kicked off around 2020 as the first AI-enabled crime stories began to hit the headlines. Yes, AI-enabled crime has been happening since well before the big ChatGPT watershed. In fact, we can go back to mid-July of 2019 when Cyber Security Hub journalist Alarice Rajagopal reported on a press release that, AI could escalate a new type of voice phishing cyber attack.
Just six short weeks later, on the 30th of August, the Wall Street Journal vindicated these concerns. Catherine Stupp broke one of the first big AI-enabled cybercrime stories on fraudsters using AI to mimic a CEO’s voice. After that, the floodgates opened and the stories of generative AI heists, cons, spoofs, deepfakes, extortion, infringements and more poured in from around the world.
These headlines inspired our first three Bot or Not? perception tests: the Text Edition in 2023, the Speech Edition in 2024, and the Music Edition in 2025. But we haven’t stopped there because…
Then things got complicated
One of the biggest cases of recent times is the Arup heist. (This is such a spectacular example that you can listen to a whole en clair podcast episode on it.) What stands out in this case – aside from the amount of money involved – was their decision to use emails, text messages, and then live audiovisual interaction.
In different words, whilst “static” content like an email or a voicenote is interesting, the future of generative-AI-enabled crime is very likely to be found in its ability to convincingly interact with a target, from holding a casual conversation to giving C-suite-style orders to make big transfers.
So what now?
That takes us to our ongoing suite of current work, which includes…
- The Turing Editions: Interactive tests that challenge both humans and AI to see who is better at Bot or Not? when trying to detect whether their conversational partner is human or AI
- HackaCon 2026: An online hackathon challenge to see whether it’s possible to spoof specific target voices taking part in natural human conversation – nowhere near as easy as it sounds…
- Fort Vox: A little voice-protected locked box that tech-minded kids (and adults) can build to get them thinking about digital identity, voice biometrics, red-teaming, and security