ChatGPT is generating fake news stories — attributed to real journalists. I set out to separate fact from fiction

Of interest (I am starting to find it useful as an editor):

“Canada’s historical monuments are also symbols of Indigenous genocide.”

“Police brutality in Canada is just as real as in the U.S.”

Those seemed to me like articles that my colleague, Shree Paradkar, a Toronto Star social and racial justice columnist, could have plausibly written. They were provided by an AI chatbot in response to my request for a list of articles by Paradkar.

The problem is that they don’t exist.

“At first blush it might seem easy to associate me with these headlines. As an opinion writer, I even agree with the premise of some of them,” Paradkar wrote to me after I emailed her the list.

“But there are two major red flags. The big one: they’re false. No articles I wrote have these headlines. And two, they either bludgeon nuance (the first headline) or summarize what I quote other people saying and what I write in different articles into one piece,” she said.

Paradkar’s discomfort reflects wider concerns about the abundance of fake references dished out by popular chatbots including ChatGPT — and worry that with rapidly evolving technology, people may not know how to identify false information. 

The use of artificial intelligence chatbots to summarize large volumes of online information is now widely known, and while some school districts have banned AI-assisted research, some educators advocate for the use of AI as a learning tool.

Users may think that one way to verify information from a chatbot is to ask it to provide references. The problem? The citations look real and even come with hyperlinks. But they are usually fake.

In recent months, academics have issued multiple warnings that ChatGPT was making up academic studies, including convincing scientific research abstracts. This came to the attention of Oxford University professor David Wilkinson when a student turned in a paper with a reference to a study that he couldn’t locate, but which was similar to fake references he found on ChatGPT.

It is less well known that media sources provided by chatbots are often fabricated as well. The Guardian recently called attention to the confusion that ensued at the newspaper when a reader inquired about an article that did not appear on The Guardian’s website.

The headline was so consistent with the newspaper’s coverage that staff thought it could have been something the reporter had written in the past. Staff went deep into computer systems to try to track down the article in case it was deleted. Luckily before more time was wasted, the reader disclosed that the reference came from ChatGPT.

“The invention of sources is particularly troubling for trusted news organizations,” wrote Chris Moran, The Guardian’s head of editorial innovation.

“It opens up whole new questions about whether citations can be trusted in any way, and could well feed conspiracy theories about the mysterious removal of articles on sensitive issues that never existed in the first place,” said Moran.

In cases of both fake media and academic references from chatbots, this is because “generative” AI chatbots are large language models. They are trained on a huge body of internet data, and can reproduce human language. But the important point is that they are not search engines, said Subodha Kumar, professor of statistics, operations and data science at Temple University in Philadelphia.

“The general public using ChatGPT now doesn’t really know how it works. It creates links and references that don’t exist, because it is designed to generatecontent,” Kumar told me.

“That’s what makes AI beautiful, but also problematic. False references can fool the public and policymakers, so if we have a newspaper article based on a false reference, then we’ll be in big trouble,” he added.

I wondered if ChatGPT has undergone any modifications since researchers raised concerns about the prevalence of fake citations. The U.S.-based company Open AI did not reply to my request for clarification. So like many other curious journalists have done, I interacted with the bot itself.

“Could you give me some articles by Globe and Mail journalist André Picard?” I asked.

“As an AI language model, I do not have access to specific articles or publications by individual authors, including André Picard, as I do not have browsing capabilities or access to the internet. I can, however, provide you with a general format for citing articles,” it replied.

This response appeared to clearly explain ChatGPT’s limitations. I was pleasantly surprised.

But alas, when I asked ChatGPT for a summary of Huawei executive Meng Wanzhou’s legal battle in Canada and asked it to provide sources, the list of references it provided were all fake.

The hyperlinks were either broken or led to unrelated pages. A note at the bottom said that since it was trained on information up to September 2021, it may not reflect current events. But there was no disclaimer that the references themselves were fake.

“Why are your citations fake?” I asked ChatGPT.

“As an AI language model, I do not generate fake citations deliberately. However, I do not have direct access to external sources or the internet, and I do not provide real-time information … It’s always a good practice to cross-check and verify information from reputable and reliable sources independently, especially for critical or sensitive topics.”

But Kumar said it is very possible for developers to train chatbots to verify the references they generate.

“They don’t have to fundamentally change the concept to be just like Google Search, but the algorithm can be easily improved by giving it one more step: When references or links are created, check for the veracity of the content.”

Kumar added that other companies may soon offer new AI products that provide more reliable references, but as a “first mover” in the field, OpenAI has a special responsibility to address the issue.

OpenAI has said it is aware of the potential of generative AI to spread disinformation. In January, the organization partnered with Stanford University and Georgetown University to release a study forecasting potential misuses of language models for disinformation campaigns.

“For malicious actors, these language models bring the promise of automating the creation of convincing and misleading text for use in influence operations,” the study found.

And ChatGPT is only one out of a plethora of chatbot products from different companies, including apps that purport to be based on ChatGPT’s open API. I had found the list of my colleague’s fake opinion articles on one such Android app, “AI Chat by GPT,” (ChatGPT doesn’t currently offer a mobile version.)

For Ezra Levant, a conservative Canadian media commentator, the app offered up fake headlines on hot-button issues such as a fake column alleging that global migration will “undermine Canadian sovereignty” and another that Prime Minister Justin Trudeau’s carbon tax is in fact a “wealth tax.”

Paradkar pointed out that the generation of fake stories attributed to real people is particularly dangerous during a time of increasing physical violence and online abuse against journalists worldwide.

“When AI puts out data that is incorrect but plausible, it counts as misinformation. And I fear that it offers ammunition to trolls and bad actors confirming their worst biases and giving them more reason to abuse journalists.”

Source: ChatGPT is generating fake news stories — attributed to real journalists. I set out to separate fact from fiction