College student claims to develop an app that can detect essays written by chatbot ChatGPT

A 22-year-old college student claims to have created an app that can tell whether content was authored by ChatGPT, the controversial chatbot that has caused concerns about academic plagiarism. Over the summer, senior at Princeton University Edward Tian created GPTZero. Within a week of its inception, it received 30,000 hits. Tian claimed that the goal was to address how fast and reliable academic writing can be used to circumvent anti-plagiarism software and cheat in tests. His initial tweet went viral, with more than 5 million views, and stated that the program could "quickly and efficiently" determine whether an essay had been written by artificial intelligence.

I spent New Years building GPTZero — an app that can quickly and efficiently detect whether an essay is ChatGPT or human written

— Edward Tian (@edward_the6) January 3, 2023

Since then, Tian has received hosting and memory help from Streamlit, the free platform that runs GPTZero, to keep up with web traffic. The program runs calculations of "burstiness" and "perplexity," which assess phrase variety and complexity, respectively, to see if content was produced by artificial intelligence. The likelihood that the text was generated by AI increases with how familiar the text is to the bot, which is trained on comparable data. Tian informed subscribers that although the new model followed the same principles, it had a better ability to identify content that contained artificial intelligence.

here's a demo with @nandoodles's Linkedin post that used ChatGPT to successfully respond to Danish programmer David Hansson's opinions pic.twitter.com/5szgLIQdeN

— Edward Tian (@edward_the6) January 3, 2023

“Through testing the new model on a dataset of BBC news articles and AI generated articles from the same headlines prompts, the improved model has a false positive rate of < 2%,” he said.

“The coming months, I’ll be completely focused on building GPTZero, improving the model capabilities, and scaling the app out fully.”

Toby Walsh, Scientia professor of artificial intelligence at the University of New South Wales, wasn’t convinced.

He said unless the app was picked up by a major company, it was unlikely to have an impact on ChatGPT’s capacity to be used for plagiarizing.

“It’s always an arms race between tech to identify synthetic text and the apps,” he said. “And it’s quite easy to ask ChatGPT to rewrite in a more personable style … like rephrasing as an 11-year-old.

“This will make it harder, but it won’t stop it.”