I recently wrote a short story within the span of roughly 24 hours.
Usually I overthink it. Usually it takes more time for me to finish any given piece, whether fiction or not. Even this short story required me to brush up with some minor research. But it flowed so easily, so naturally, that I started writing one night and finished the next.
The process of writing it made me genuinely happy. I was proud of my finished little story.
After all was said and done, I got a funny curiosity. I wondered what the results would be if I ran it through an AI detector. So, I did just that.
I started with ZeroGPT.
My story scored 29% AI-generated.
It flagged entire paragraphs in some areas. Chunks of the story that had poured out so naturally were now highlighted, confidently deemed to be machine-generated words.
What the hell?
I started to feel a strange feeling, mixed with something like identity confusion.
Do I just write like a bot?
I remembered how some months back, I found old essays and stories in my sent emails. They were years-old writings, sent and dated before AI was what it is. Reading them recently was a weird experience.
I noticed so many em-dashes that I’ve since trained myself not to use as much, even though it was once normal for me. I noticed how essays I wrote in 2016 used the word “foster,” a no-no word I’ve since avoided using. I noticed many things that I filter out of my communication now, because it might signal that my writing isn’t my own.
My short story didn’t have any of those things either.
What was it about me that screamed “bot” to the detector?
The result bothered me. So, I ended up trying another experiment.
I had an actual bot rewrite my story entirely.
I sent my finished story and asked the bot to rewrite the same story, in a completely new way, in any style the bot wanted.
Then, I pasted the AI rewrite into the same AI detector, ZeroGPT.
The AI’s result?
24.2% AI-generated.
An AI’s writing scored lower for AI than my own original story.
How was a bot more human than me, a real human?
These detectors are unreliable. I knew that. Still, it bugged me. I ran the two versions through detectors that are thought to be more accurate.
GPTZero (similar name; different detector) labeled my story with high confidence as 100% human. The AI rewrite received moderate confidence that it was AI generated, with 75% pure AI and 25% mixed.
Pangram detected with high confidence that my story was 100% human written. The AI rewrite was deemed mixed, with a 61% AI score and 39% human score, all rated with high confidence.
Originality.ai was 100% confident that my story was original. The rewrite received 79% confidence that it was AI.
These results restored some of my sanity. They could all accurately tell that I was entirely human. No cyborg here.
But the AI rewrite’s results were interesting too.
With the initial test on ZeroGPT, the AI rewrite scored more human than mine. Even the more accurate tests could label the vast majority as AI with confidence, but the mixed results are interesting. Maybe those are areas where an original human idea grants a small bit of legitimacy to AI text?
I don’t know. But at least for now, the “Are you human?” checks have been passed, and the identity crisis is managed.
P.S.: It turns out that I have a curiosity problem, or maybe a paranoia problem. You never know, even in a conversational piece it’s totally possible to sound like a cyborg. So I ran this through detectors too. Results are below.
- ZeroGPT (our unreliable first test): 7.3% AI.
Ironically, it didn’t flag anything after the line that first mentioned ZeroGPT.
Now for the more accurate detectors…
- GPTZero: 100% human, high confidence.
- Pangram: 100% human, high confidence.
- Originality.ai: 78% confidence of human origin.
I’m still shaking my fist at the last result, but I guess you can’t win ‘em all. At least I have a fist to shake.
Leave a Reply