Trying to help you detect AI written content in 2023
Artificial intelligence (AI) chat bots have been around for quite some time. Heck we can go back to 1997 when the chatbot Jaberwacky was released online to the world. Yes it was rather basic and strange to talk to, but it worked and was quite impressive back then. Since then we have come a long way.
Only recently did we start to see such huge leaps in this kind of technology. First we had AI writing tools that would write articles for you, then google started to work on Large Langue models (LLM for short) and then we have the release of ChatGPT. This is when things started to get a little out of hand. People were amazed at the way ChatGPT would respond to it’s users, some even thought this was an example of AGI (it’s not but it’s pretty advanced machine learning mixed with neural networks and deep learning).
This has lead to AI-powered systems being used over a wide collection of businesses, some of these businesses sole purpose is just to provide users with AI written content. It has taken the blogging industry by storm, this of course has lead to concerns around this type of technology and if it would be write to create this type of content. Now more than ever it is so easy to create AI written content.
AI content in and of itself isn’t actually all that bad. It is also not against Google’s policies. As long as your AI written content is formatted correctly, provides good information and is of course a joy to read. Well majority of AI content out there is not a joy to read, it is awful and not very Human like. So we have seen a mass spam of crappy written content all over the internet lately, because of this companies have been working hard to detect poorly written AI content in a way to battle this recent content plague.
The last thing anyone wants is for people to misuses AI written content, so these detectors are a good way to ensure that people are not abusing the system. Maybe some day in the future google might implement it’s own detection methods to ensure this system stays above board.
In this article, we will take a look at the best FREE AI detection tools on the market. We don’t expect you to pay for this kind of thing because currently there is no evidence if it actually makes a difference. Remember AI content detection is still in it’s infancy and they can not be treated as fully reliable sources. But it is a good place to ensure the content you have written is not “bot like”.
Our Sample Text to Check Against AI Detectors
In order for us to perform a fair comparison. We are going to run two prompts one is the standard simple prompt the other is a prompt engineered chat. We will be using ChatGPT to create these AI Generative responses as it’s the most advanced chat bot to date. Making this a good test for our AI content detectors.
Prompt 1 – Standard Basic Prompt:
“Write me a 200 word blog post on Super Cars”
Here is the response ChatGPT has come up with for us you can see the full reply that has been edited for input here.
A click glance at this you might just see a big issue with this response. That issue is that every new section starts with the word “Super cars” making this poorly written.
Prompt 2 – More Advanced Prompt:
“You are a car mechanic with 20 years worth of experience in the field. You have worked on countless super cars over the years so know your stuff. You are a big fan of Ferrari and Lamborghini but still know a lot about all car models. I want you write a 200 word blog post on super cars. Write the blog post in a witty yet intelligent manner and make sure you throw at least one joke about cars into the post, this blog post is written for an audience of middle aged men around the age of 40 to 50 so try to target that audience with your response”
Sure it’s more advanced, but that’s the point we want to try to fool an AI detector, let’s make it a bit of a challenge for our test services. You can see a picture of the response below if you want to see the full response check here.
We are going to use these two examples in the AI content detectors to see if they can do exactly that detect AI content still in 2023.
Now Let’s Test out the AI Content Detectors for 2023
We are going to test out 3 AI content detectors. Remember these are all free to use.
AI Text Classifier: OpenAI’s Native AI Detector
Price: Free
Rating: 1/10 – It’s not very good, not very good at all.
AI Text Classifier ok considering we have created this content using OpenAI’s ChatGPT it makes sense for us to try OpenAI’s AI-generated text detector. In theory this should be spot on as they know exactly what the algorithms are behind creating these responses. Right? well, not so right. The company has admitted that the classifier used is only partially accurate, seems odd no? Basically they state that you should not rely on it’s responses entirely. So we are not off to a good start.
I have used the tool in the past and it has mistakenly identified human written content as AI-generated but partially I believe this was down to the content written was rather bland. There are a few ways you can beat content detectors and the person who wrote the content I was checking seemed to be a human with a bot like brain.
This classifier will fail if you use small amounts of text so it’s advised that you don’t use less than 1000 characters in your input (150 – 250 words). Sine our posts are around 300 words this is plenty of information to feed this detection tool.
Now these results might surprise you, we think this tool is by far the worst at detecting AI content, it’s not the fact that ChatGPT is amazing at creating written content, it’s the fact this tool is useless.
Simple Prompt Results:
Here are the simple prompt results, we would expect it to be able to detect this form of AI written content as it came direct from ChatGPT.
Now this was the basic prompt, nothing was fed to the chatbot it was quite basic in it’s nature. So for us to receive the message “very unlikley AI-generated” was a bit of a surprise. OpenAI can not detect it’s own AI written content. That’s not a good sign at all.
Advanced Prompt Results:
Here is the results for the advanced prompt, we would expect this to be harder to detect as it has added character, experience and style to the prompt.
Ok, a little bit of a surprise it has no changed it’s tone. It is saying “unlikely AI-generated”. Still not able to detect weather it is or not though.
The biggest flaw with this type of AI detection is that it is based off, of a set of rules. So it looks for certain things within a document and add points up. When it reaches a certain threshold it will state weather or not it’s a bot. The bad side about this is that once you know what the algorithm is you can easily bypass it by not using certain keywords. The use of synonyms can quite easily beat this type of AI detection tool.
Please note though you can paste over 1000 words into this AI content detector. So perhaps if you wore to write a longer article it might show different results. Worth looking into but probably still rubbish at detecting AI created content.
GPTZero
Price: Free
Rating: 3/10 (it used to be great)
GPTZero this has had some use within the education sector, where teachers have been using it to catch pupils that are writing AI written content for their papers. There has been a bit of an issue when it comes to students creating content with AI. It’s not ethically correct and can result in you being suspended from your course. We do not condone of this behaviour, so if you are doing it can you please stop.
GPTZero is a free tool that anyone an use. It checks for a wide range of things within written text. Some people say this can be a problem within its self because doing so limits the use of the AI content detection tool. As ChatGPT improves it is going to change the way it works, so GPTZero needs to be constantly updated and upgraded to keep up with this.
This tool was created by Edward Tian, a Princeton University computer science and journalism student. It was specially created to check for ChatGPT written content so it should be able to detect our prompts right? well let us see
Simple Prompt Results
This system has been designed to detect GPT4 written content this what ChatGPT is currently using. Let’s see the results:
Ok so it didn’t detect any AI written content, disappointing. This was a straight up copy and past, wonder why it’s not flagging anything up at all. Again let’s try the advanced prompt.
Advanced Prompt results
Let’s see if this one is any different:
Nope, still think it’s written by human. Looks like the algorithm need updating on this ne. You see the real issue we always have with Detecting AI content is that these Chat Bots are getting much more better at providing human like content. So at some point maybe these tools might become obsolete. Or perhaps some companies might add hidden notifiers that let people know it’s written by AI.
Now let’s head over to our final test
Content At Scale: AI and Plagiarism Detection
Price: Free AI detection only (This is the one we personal use)
Rating: 7/10
Content At Scale is another AI content detection tool. It offers other services that are paid for but for this case we are just using their AI written content detector.
The way this tool works is slightly different in it’s response. It uses three primary conditions when analyzing text. These are predictability, probability, and pattern. It also uses Copyscapes technology to perform high quality scans for any unoriginal or copied content.
We have been using this tool for the past few months, it has been quite reliable in detecting low quality content, but has become worse of the past few weeks. This is partly to blame for ChatGPT becoming more efficient through deep learning and the fact these tools might not be updated as frequently as we would like.
Anyways on with our final test, we are using the same prompt like we have done through the other AI content detection tools.
Basic Prompt:
Let us see the results
Hey look we found some AI written content. Ok fair enough it thinks some of the content is human written but it’s a start no? The thing we like the most about this AI Content detector is that it shows is parts that it has detected. It then colour codes them on severity. Red been the most obvious detection of AI written content.
Please note though this is not a definitive way of detecting AI written content it is just an AID. It may also pick up poorly written content and say it was written by AI.
Advanced Prompt:
Now let’s see if an advanced prompt can fool this type of detection
As you can see from the above image it has detected AI written content and given us a slightly improved score. Sure the score seems a little on the high side, but this does show that some Online AI content detection tools can still detect AI written content that was created with ChatGPT!
Conclusion
Sure this wasn’t the best test in the world. All we have used is a 300 word response from ChatGPT with two different variations of prompts. We could have gone more in depth and created a much larger article to work with or done multiple articles. But why spoil the fun? why not create your very own articles with ChatGPT run them through these detection tools and let us know if you managed to find AI written content, let us know in the comments below.
At the end of the day AI text generation tools are getting better on a daily basis. It’s only a matter of time before they start sharing content that seems human like. This isn’t necessarily a problem if the content written is of high quality.
We will always have the issue of people creating AI content and then re writing it in their own style anyways, so how do you know even this article was not created with AI? (91% human 😉 )
