What Happened When We Got ChatGPT to Pitch Blog Post Ideas

ChatGPT has taken the academic and creative worlds by storm, but how well can it do with a subject as complex and rapidly changing as wireless retail?

When you consider the seismic shifts this new technology is causing in academia and marketing, it’s hard to believe that the first open release of ChatGPT – Open AI’s free chatbot that easily and quickly generates human-sounding writing – happened on November 30. And yet, universities are scrambling to get policies in place to deal with the inevitable flood of AI-written assignments and marketers are alternately enthusing (it’ll save us time!) and fretting (it will replace humans!) about ChatGPT’s copywriting capabilities. 

ChatGPT-written marketing is already out there. Mint Mobile recently released an ad written by ChatGPT that was… surprisingly good.

So how well would it do with wireless retail?

The Brief: Generate Blog Post Ideas For Wireless Retail

After some experimentation with prompts, we fed ChatGPT a series of prompts about some key trends and topics in the telecom retail industry. The results were… pretty mixed:

• About a fifth (22%) of the ideas it generated were sufficiently interesting and topical to our audiences to earmark as potential future blog post topics.
• Of particular concern – 11% were either factually incorrect or outright nonsense.
• Another quarter (25%) of results were either too generic, too outdated, or hype without substance.
• One tenth (19%) were irrelevant to our audience.
• The remaining 32% were… fine? They weren’t too hype, or too 101, or too generic, but they also just weren’t very exciting.

What Did It Get Wrong? And Why?

While we won’t break down every result, it’s interesting combing through some of the worst topics it generated, as it highlights the obvious challenges and limitations of ChatGPT.

Example #1: ​“The rise of the budget smartphone: How affordability is driving global sales”

Problem: Affordability is no longer the primary factor driving smartphone purchases.

ChatGPT’s training data only goes as far as the end of 2021, which means that it’s unable to comment on trends that have developed after that. However, as mentioned in our recent post, the current trend is for higher average selling prices as consumers are increasingly wanting to buy phones they won’t have to replace every two years – with ASPs hitting the highest level since 2011.

Example #2: ​“The impact of 5G technology on smartphone sales: What consumers can expect in the coming years”

Problem: 5G was seen as the next big trend in wireless two years ago, but now it’s old hat. 

It’s going to be a long time before North American consumers see any real impact from 5G. So much so that the lack of conversation about 5G at this year’s CES was almost a trend in itself. Given that the hype over 5G has largely failed to materialize, we definitely aren’t going to prioritize any of the several 5G topic suggestions ChatGPT gave us.

Example #3: ​“The rise of cloud-based retail software solutions: How they are transforming the industry”

Problem: Cloud-based retail software isn’t ​“transforming the industry” because it’s been the standard for years.

2016 called, and it wants its generic hype back.

The Verdict: ChatGPT has its uses, but needs to have human hands on the wheel

ChatGPT is useful for topic ideation if you don’t mind throwing a lot of spaghetti at the wall first. And while some marketers have touted its capabilities for writing topic summaries and social post copy, we’d say the need for copywriters isn’t going away any time soon.

ChatGPT In Its Own Words (With Lots of Red Pen)

We asked ChatGPT to write the conclusion to this post, but even when it was fed detailed points as well as tone prompts, the copy we got back was… well. A mess that sounded like a high schooler wrote it:

So instead, here’s ChatGPT’s conclusion with our edits:

“ChatGPT is an impressive tool for generating human-like text, but at the end of the day it can only give us insights based on the data it’s been trained on. It’s not capable of the kind of intuition, creativity, and critical thinking that humans bring to the table, and it’s not a replacement for human expertise.

“When it comes to identifying and reporting on leading trends in complex and rapidly changing market segments, human insight is still incredibly valuable. Humans have the ability to understand context, interpret data in new ways, and make connections between seemingly unrelated pieces of information. It’s our unique perspective and ability to think outside the box that will always be essential in making sense of these complex market trends.”