It’s impossible to have missed the buzz around ChatGPT, and honestly speaking, the AI is worthy of all the media attention. It’s a massive improvement from the pre-programmed response text-bots of the 90’s era. AI based start-ups & systems have been steadily increasing in quality and number. So much so that you’re now almost spoilt for choice if you’re looking for an AI blog writer or illustrator. But are systems like ChatGPT really as good as they appear? Should journalists and content writers be crying into their beer?
Why is ChatGPT so good? How does it actually work?
To understand how ChatGPT works, we should start by asking ‘why is it so convincing?’. ChatGPT is a language model, which means it’s very good at stringing words together in ways that humans would do. Put simply, it writes the same way a real person does. Very confidently. It’s that natural and confident way that it communicates which makes it so convincing.
The answer to any given question you ask it, will always be different, because it’s answers depend on a probability distribution model. It’s constantly changing based on what’s it’s learned, and like a dice roll, always likely to be different.
Unlike a person, when ChatGPT ‘learns’ it doesn’t think critically, interpret or reason, it just finds patterns. It uses those patterns to generate a plausible (based on probability) sounding answer to a question. In other words, it’s able to create Grade A BS.
Is the content as good as it seems?
Systems like ChatGPT create very convincing content, often even with what appear to be citations. It’s so good that several major publications recently started using AI to write entire articles, with human editors to help fact check. Without a human author though, things can start go wrong quite quickly, as CNET found out the hard way. Human authors usually do baseline homework to ensure their content is mostly correct before sending to the editor. If the content is written by an AI, the editor needs to be prepared to scrub the content a lot more thoroughly to prevent issues.
To test how accurate ChatGPT really is, I asked it to ‘create a list of inspirational quotes’. The output was fantastic and I was duly impressed by the quotes. Unfortunately perhaps for ChatGPT, I asked it the same question a few different times, and that’s where the cracks started to appear.
On closer inspection of the ‘quotes’ it was pretty clear that things were a little off. It had given me two identical quotes, but with different attribution. The quote didn’t really sound like it would come from either of the people it’d suggested. Unsurprisingly, a quick Google search revealed that both the attributions were wrong. The quote is from neither Walt Disney, nor Melinda Gates, it is in fact from Watts Humprey.
AI responses are based on patterns and probabilities, not reason and understanding. No matter how good they may appear to be, they need a human to apply critical thinking and validation. In practice this means, you can’t outsource your entire content production to an AI system (not yet anyway). It might be a great tool for inspiration, content outlines or even ‘fact’ finding, but that’s it. Any content it creates by itself may be riddled with very eloquently worded errors.
That isn’t to say that there aren’t an enormous number of great AI powered tools that you can use. Just remember that no matter how advanced the tools, they’re only as good as the operator. So start warming up your AI muscles, you’re going to need to know how to use them. I’ll post a list of AI Powered tools that you might find useful in a short while.
TLDR
Language models like ChatGPT don’t do critical thinking or reasoning, they provide nicely written but potentially unsound output, that is based on patterns & probability. Some companies that dived in head first with completely AI written content are paying the price for this retrospectively.
It’s not good enough to replace a human author yet, but they do make life easier. It’s a good idea to get familiar with how these tools work, and figure out how they can complement your workflow. They’re likely going to get better, and are almost certainly here to stay.