AI vs Human Content Creation: An SEO Experiment and Analysis

Jay Clair
6 min readJul 14, 2023

--

Welcome to my continued deep dive into the world of AI content creation. Today, we’re exploring an exciting frontier where the realms of technology and creativity intersect. Yes, you guessed it right! We’re talking about the grand face-off: AI vs. Humans in the arena of content creation.

Now, if you’re imagining a sci-fi movie scene with robots and humans locked in an epic battle, hold your horses! This isn’t about world domination (at least, not yet 😉). It’s about a fascinating study conducted by our friends at Reboot Online. They’ve rolled up their sleeves and jumped into the nitty-gritty of SEO content creation, comparing the prowess of AI and human writers.

AI vs Human Content Creation
Midjourney V5.2: AI vs Human Content Creation

The Study: A Quick Peek

Reboot Online, in a bid to answer the age-old question (okay, maybe not age-old, but certainly a hot topic in our digital age), conducted a study. The question? “Who writes better content for SEO rankings — AI or humans?”

To answer this, they set up an experiment, a bit like a digital bake-off. They created a new, artificial keyword (because why not add a dash of mystery to the mix?), and built test domains filled with content written by both AI and human writers. It’s like a reality show but for content. Exciting, isn’t it?

So, buckle up, dear reader! We’re about to embark on a journey to explore this intriguing study, its findings, and of course, our two cents on the methodology.

Stay tuned, and let’s dive right in!

Overview of the Study

Alright, let’s roll up our sleeves and dive into the details of this study.

The Purpose of the Study

The folks at Reboot Online had a burning question: “Who’s the better writer for SEO rankings — AI or humans?” Now, that’s a question that could spark a debate at any digital marketing conference. So, they decided to put it to the test.

The Methodology: A Digital Bake-Off

To answer this question, they set up an experiment. Picture a digital bake-off, but instead of pastries, we’re dealing with content. And instead of chefs, we have AI and human writers.

They created an artificial keyword, “flemparooni” (I know, it sounds like a delicious Italian dish, doesn’t it?). The idea was to have a keyword that was a clean slate for Google, a word that had never been seen before by the search engine.

Then, they built 10 test domains. Five of these domains were filled with content written by AI, and the other five were filled with content written by human writers.

The Contestants: AI vs. Humans

On one side, we have AI, the new kid on the block, all shiny and filled with algorithms. On the other side, we have human writers, armed with creativity, experience, and an uncanny ability to consume copious amounts of coffee. ☕

Now that we’ve set the stage, let’s move on to the results. What happened when AI and humans went head-to-head in this digital content creation showdown?

Key Findings

Alright, let’s get to the juicy part — the results!

The Winner: Humans…For Now

Drumroll, please… The study found that human-written content ranked better than AI-generated content. The average ranking for AI-generated domains was 6.6, while human-written domains had an average ranking of 4.4. So, for now, humans take the crown! But remember, in the world of SEO, the game is always on!

The Underdog: AI Content Detection Tools

Here’s a surprise twist — third-party AI content detection tools have improved significantly over the course of the experiment. They’re now able to detect GPT-4 generated content much more reliably than they were able to when the experiment first started. It’s like the little engine that could, steadily chugging along and getting better and better.

The Real MVP: Skilled Human Writers

The study suggests that content written by skilled human writers, dedicated to creating content that resonates and connects with their target audience, is still in high demand. It’s like a good old-fashioned home-cooked meal — there’s just something about it that can’t be replicated. 🍲

So, there you have it, folks! The results of the grand face-off. But wait, there’s more! Let’s take a closer look at the methodology of the study. After all, the devil is in the details, right?

Critical Analysis of the Methodology

Now, let’s put on our detective hats and delve into the methodology of the study. 🕵️‍♂

The Artificial Keyword: “Flemparooni”

First up, the use of the artificial keyword, “flemparooni”. While it’s a fun word to say (try it, it’s quite catchy!), it’s not exactly representative of real-world SEO. It’s like practicing your tennis swing with a baseball bat — it might help, but it’s not quite the same thing.

Real-world SEO involves complex and competitive keywords, and the performance of AI vs human content might differ in such scenarios. Plus how does Google even rate a new topic from its Help Content Update and how on earth does one demonstrate EEAT for “flemparooni”.

The Sample Size: A Small Party

Next, let’s talk about the sample size. The study used a small sample size of 10 test domains (5 AI, 5 human). While this is understandable given the resource-intensive nature of such experiments, it’s a bit like trying to predict the weather for the next month based on one day’s data. A larger sample size could provide more robust and generalisable results.

The Speedy Gonzales: Rapid Advancements in AI

The field of AI is rapidly evolving — it’s like trying to keep up with a cheetah on a treadmill. The study acknowledges that significant advancements have occurred even during the course of the experiment. This means that the results might not hold true for newer, more advanced AI models.

The Missing Piece: Human Editing of AI-Generated Content

Last but not least, the study did not consider the impact of human editing and refining of AI-generated content on rankings. It’s like making a movie but forgetting about the editing process. In real-world scenarios, AI-generated content is often edited by human writers to align it with the brand’s tone of voice and style, which could potentially improve its SEO performance.

So, while the study provides some interesting insights, it’s important to remember that it’s just one piece of the puzzle. The world of SEO is a complex one, and there’s always more to learn and explore. So, let’s keep digging, shall we?

Implications for Content Creators and SEO Specialists

Now that we’ve dissected the study, let’s talk about what it means for us, the content creators and SEO specialists.

The Demand for Human Touch

Despite the rapid advancements in AI, the study suggests that there’s still a high demand for content written by skilled human writers. It’s like the difference between a mass-produced shirt and a tailor-made one — the personal touch just adds a unique value. So, if you’re a human writer, take a moment to pat yourself on the back. You’re still very much in demand!

The Role of AI

While AI might not have taken the crown in this study, it’s important to remember that it’s not a competition. AI can be a valuable tool for content creators and SEO specialists. It’s like having a super-efficient assistant who can help you with your work. So, don’t rule out AI just yet. It’s here to help, not replace.

The Future of AI in Content Creation

As we look towards the future, it’s clear that AI will continue to play a significant role in content creation. With rapid advancements in technology, who knows what the landscape will look like in a few years? It’s like trying to predict the next fashion trend — you never know what’s going to be the next big thing.

But one thing is for sure — the future of content creation is a collaborative one, where AI and humans work together to create engaging, SEO-friendly content. It’s not about AI vs. Humans, but rather AI and Humans. So, let’s embrace the future and see where this exciting journey takes us!

Stay tuned for more insights and discussions on the world of content creation and SEO. And remember, whether you’re an AI or a human, keep creating, keep innovating, and keep having fun!

SPOILER ALERT

This post was written by AI, apart from this section. I know…. Meta hey!

So what are your thoughts on the AI vs. Humans content experiment?

Do you have any insights or experiences to share?

I’d love to hear from you! Drop a comment below and let’s keep the conversation going.

Stay tuned for more insights and discussions on the world of content creation and SEO. Until next time, happy creating!

Check out the full experiment here

--

--

Jay Clair

Head of Marketing at Bluesfest. SEO Nerd & Writer. About Me: Contrarian, rational optimist, curiosity is king.