In the world of artificial intelligence, ChatGPT once reigned supreme, captivating professionals, students, developers, and creators worldwide. It seemed like everyone was turning to this revolutionary tool for assistance, with web traffic and media coverage surging like a tidal wave. However, in recent times, many have noticed a decline in mentions of ChatGPT. Some attribute it to students on summer break or mere curiosity, while others claim the hype has cooled off. So, what’s the real story behind ChatGPT’s changing fortunes? This article aims to unravel the mystery.
Analyzing the Dwindling Traffic
If you’ve read articles like “ChatGPT Predicts 900% Growth” or “ChatGPT Achieves Facebook’s Decade-Long Goals in Six Months,” you might assume that this text prediction system had everything venture capitalists dream of exponential growth. However, keen observers might have noted recent articles like “Ugly Numbers from ChatGPT Reveal a Shrinking Demand for AI” and similar reports from multiple media outlets.
First, let’s take a look at OpenAI.com’s declining web traffic:
Since May, OpenAI’s traffic has dropped by 29.15%. Several theories attempt to explain this phenomenon, but it doesn’t necessarily indicate a decrease in interest or usage of generative AI across various industries:
Theory A) Professional and regular usage remains high or is still growing; the decline is among new or non-specific task users.
Theory B) A substantial portion of users is related to educational purposes, leading to the illusion of decreased usage during summer vacations in the United States, Canada, and other countries.
According to data from Datos, we not only have access to OpenAI’s monthly traffic but also the number of visits to these pages. This should help us validate or disprove Theory A.
Datos provides detailed monthly traffic data for all OpenAI panels since September of the previous year (2022). The distribution chart looks as follows:
Here, we can see that there has indeed been a decrease in monthly visits by users who access ChatGPT 1-2 times per month. This group’s visitation had also significantly dropped after December of the previous year when explosive news about ChatGPT 3.5’s capabilities drove rapid initial adoption. However, when analyzing the groups with monthly visits of 3-10 times and over 11 times, Theory A no longer holds water. Both of these groups have shown noticeable declines in usage since May. In fact, the group with over 11 visits per month has been steadily decreasing since April!
With Theory A debunked, we’re left with Theory B: Users in the education sector are primarily responsible for the decline in ChatGPT traffic. Looking at the decline data from April onwards, this is a challenging claim to substantiate, but the process of uncovering answers sheds more light on ChatGPT usage patterns, so let’s delve further.
Programming: ChatGPT’s Leading Use Case
Is it the users in the education sector driving ChatGPT’s applications? Are we grooming a generation of students to rely on AI for most of their tasks? Conversely, is ChatGPT replacing the need for Google searches to answer questions, eliminating the necessity for software programmers, or taking over as game masters in role-playing games?
To answer these questions, Datos provided over 7,000 real user prompts from ChatGPT, and we selected the most credible and relevant 4,098 prompts (eliminating those with only a few words, gibberish, emojis, or explicit content). The results were intriguing.
First, let’s break down the number of prompts per session:
From the chart above, it’s clear that ChatGPT users are almost evenly distributed between single prompts, 2-4 prompts, and sessions with 5 or more prompts (each accounting for approximately one-third). However, this kind of analysis doesn’t tell us what users are doing with these prompts.
Since Datos can provide full-text pages from ChatGPT, we analyzed these pages using one of the best topic classification systems available: ChatGPT’s built-in system.
I first requested granular classification from ChatGPT and selected the most common ones (out of 4,098 unique prompts, around 20 common prompts covered over 95% of the prompts), manually categorizing them into different categories as shown in the chart above.
Programming emerged as ChatGPT’s most significant use case, accounting for 29.14% of prompt series. It’s also the clearest and least ambiguous. To confirm the accuracy of the classifier, I manually checked over 100 prompts from each prompt series marked under this category, and I found programming assistance in every prompt marked as such (including tasks like writing specific code, formatting code, and debugging code).
As many have frequently noted, the tool excels in programming-related tasks, explaining its immense popularity.
Next is the Education category, which encompasses not only primary or secondary education but also personal knowledge or interest exploration and professional knowledge searches for work purposes. Content creation falls under this category as well, with some clearly being personal endeavors, such as “D&D Dungeon Master needs riddles or quests for adventures,” while others are professional, like “Write a 500-word blog post about Detroit pipeline issues” – presumably a content marketer tired of writing their materials.
Sales and marketing use cases overlap with content creation, but I’ve chosen to separate them to examine those dialogues that can only be classified as aiding sales and marketing professionals in completing tasks. These include questions related to data analysis, product promotion channels, ad optimization tasks, and even information/promotion assistance in this dataset.
For a more detailed breakdown of this investigation, I provide an almost exhaustive list of subcategories (except for some highly overlapping/subjective subcategories, which I’ve merged together):
I’ve used color coding from the pie chart above to make this breakdown more visually accessible. For instance, “Writing Help,” “Personal Content Creation,” “Creative Ideas,” and “Professional Content Creation” are all coded in gray as they fall under the broader “Content” use case.
Higher education, primary education, and homework together constitute around 10% of all use cases. However, this isn’t sufficient to explain the 29% decline in traffic from April/May to July. Therefore, Theory B seems less likely to be the sole cause.
I also found it interesting to analyze some of the most common words in ChatGPT prompt conversations. For those curious, here’s a visualization chart:
Words like “write,” “create,” and “list” are expected prominent verbs in ChatGPT prompt conversations. But finding “search engine optimization” in 2.39% of prompt dialogues is surprising!
Another pleasant surprise is “game,” accounting for 4.66% of prompts. Some other intriguing but less frequent terms that didn’t appear in the previous breakdown, yet are still interesting to me, include:
- Judge 0.61%
- SaaS 0.56%
- Pricing 0.54%
- Courses 0.46%
- Employment 0.44%
- Employers 0.39%
- Practicing lawyers 0.37%
- Lawyers 0.37%
0.34%
- Movies 0.32%
- DnD (or D&D) 0.17%
- RPG 0.15%
As I mentioned, the astonishing number of role-playing story writing and narration using ChatGPT is striking. Perhaps companies like Wizards of the Coast (a game entertainment company) should consider incorporating it in their next update for Dungeons & Dragons (DnD Beyond).
Conclusion
In conclusion, the decline in ChatGPT’s popularity is a multifaceted issue. While programming remains its most significant use case, it’s not the only one. Education, content creation, sales, and marketing also play substantial roles. The decrease in traffic can’t be attributed to a single factor, and the real reasons behind it may be a combination of Theory A and Theory B.
As ChatGPT continues to evolve and adapt, it will be interesting to see how its usage patterns change over time. One thing is clear: ChatGPT’s impact extends far beyond the programming world, and its versatility continues to intrigue and captivate users across diverse fields.