Question to the Cabinet Office:

To ask the Minister for the Cabinet Office, what assessment his Department has made of the effectiveness of the GOV.UK chat experiment.

Government Digital Service (GDS) tested an early prototype of a Large Language Model (LLM) powered chatbot (GOV.UK Chat) in December 2023. Feedback, and the quality of answers provided, were analysed from 200 users.

Nearly 70% of users found the prototype-generated responses to be useful and the answers were assessed to be accurate 80% of the time.

Ensuring the highest level of accuracy and efficiency is a priority, which is why the next phase of development and testing will focus on how to improve accuracy of the model and explore how users could best discover and use the service.