[SIGCIS-Members] ChatGPT’s Astonishing Fabrications about Percy Ludgate

[Brian Randell]

Hi:

Motivated by the extensive publicity and excitement about the new AI chatbot ChatGPT, a tool for answering natural language queries across numerous domains, I and colleagues undertook a brief assessment of its ability to answer questions on a computer history topic. To our astonishment ChatGPT responded with 48% fake facts to the twelve questions we posed it!  
A brief account of our assessment has just been accepted for publication by the IEEE Annals of the History of Computing. The full details of the assessment are to be found in the report “ChatGPT’s Astonishing Fabrications about Percy Ludgate” by Brian Coghlan, Noel O'Boyle and myself - available online at https://tinyurl.com/383pbuwb .

Cheers

Brian Randell