The BBC has discovered that AI chatbots distort and mislead when asked about current events.

Getty Images

 
The BBC's research shows that four major artificial intelligence (AI) chatbots are not accurately summarizing news stories.

OpenAI's ChatGPT, Microsoft's Copilot, Google's Gemini, and Perplexity AI were given content from the BBC website and quizzed on news questions.

According to the statement, the responses received had significant inaccuracies and distortions.

Deborah Turness, the CEO of BBC News and Current Affairs, stated in a blog that AI offered 'unending opportunities', but the companies developing the tools were playing with fire.

Our world is filled with trouble. How much time will it take for an AI-distorted headline to cause significant real-world harm?

The tech companies who own the chatbots have been contacted for a comment.
'Pull back'

ChatGPT, Copilot, Gemini, and Perplexity were asked by the BBC to summarise 100 news stories and rate each answer, as part of the study.

Journalists who were relevant experts in the article were asked to evaluate the quality of answers from AI assistants.

According to the study, 51% of AI answers to news questions were judged to have significant issues of some form.

In addition, 19% of AI answers that referenced BBC content made factual errors, such as incorrect statements, numbers, and dates.

Ms. Turness wrote on her blog that the BBC is seeking to initiate a new conversation with AI tech providers to collaborate and find solutions.

She called on tech companies to "pull back" their AI news summaries, as Apple did after complaints from the BBC that Apple Intelligence was misrepresenting news stories.

The BBC discovered some inaccuracies that included:

Gemini's statement that the NHS did not recommend vaping as an aid to quitting smoking was incorrect.
ChatGPT and Copilot reported that Rishi Sunak and Nicola Sturgeon were still in the office despite their departure.
In a Middle East story, Perplexity misquoted BBC News, stating that Iran initially displayed'restraint' and characterized Israel's actions as 'aggressive'.

Generally speaking, Microsoft's Copilot and Google's Gemini had more significant issues than OpenAI's ChatGPT and Perplexity, which has Jeff Bezos as an investor.

The BBC usually prevents AI chatbots from accessing its content, but it opened its website for testing in December 2024.

According to the report, the chatbots were unable to distinguish between opinion and fact, editorialize, and often fail to include essential context, in addition to containing factual inaccuracies.

The BBC's Programme Director for Generative AI, Pete Archer, said publishers "should have control over whether and how their content is used and AI companies should show how assistants process news along with the scale and scope of errors and inaccuracies they produce."