Charlie the CRA ChatBot

The following entry is a record in the “Catalogue of Catastrophe” – a list of failed and troubled projects from around the world.

Organization: Canada Revenue Agency (CRA)
Project type: Digital service / AI chatbot
Project Name: Charlie the Chatbot
Date: 2020–2025
Cost: $18 million+

Synopsis

Charlie the Chatbot was originally launched by the Canada Revenue Agency (CRA) in 2020. It’s goal was to help Canadians navigate the country’s complex tax system and reduce demand on the already overburdened call centers. Promoted as a modern, digital solution, Charlie was intended to answer general tax questions, improve service access, and alleviate pressure on human staff.

In 2020 Charlie was ahead of the curve. Although chatbot tools had existed for a number of years, the more mature Artificial Intelligence (AI) generative tools (such as ChatGPT) only became readily available starting in 2022. In response to those advances Charlie was upgraded in 2024 to a generative AI system so that it could respond to more complex questions.

A 2025 audit by Canada’s Auditor General examined the ability of the CRA to manage tax questions from the public. The report indicated that tests on Charlie gave incorrect answers 66% of the time. At the same time wait times to speak to a real agent averaged a little over 30 minutes (far exceeding the agency’s service standard). The report indicates that pre-release testing for the more advanced version (2024) was attaining 90% accuracy, but at the time of the audit those figures were unverified in real world operations.

Over its first 5 years of life building and operating Charlie has cost taxpayers $18 million.

Contributing factors as reported in the press

Use of immature technology in an inherently complex environment. Lack of effective quality control or operational governance. Failure to curate source data effectively before utilizing that data to power the chatbot.

Reference Links

Margot Jantz