<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:0 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Aptos;
panose-1:2 11 0 4 2 2 2 2 2 4;}
@font-face
{font-family:"Apple Color Emoji";
panose-1:0 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Segoe UI";
panose-1:2 11 5 2 4 2 4 2 2 3;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:10.0pt;
font-family:"Aptos",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:12.0pt;mso-fareast-language:EN-US">Hi:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-fareast-language:EN-US">My colleague Brian Coghlan added two further points about the dangers AI hallucinations/fabrications pose to historians who are trying to use Generative AI tools to help their investigations:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-indent:36.0pt"><span style="font-size:11.0pt;color:#212121">First, our own investigation very strongly implied the "AI error rate" was inversely proportional to the AI
<o:p></o:p></span></p>
<p class="MsoNormal" style="text-indent:36.0pt"><span style="font-size:11.0pt;color:#212121">knowledge of the query topic. So one *very easy* countermeasure would be for AI tools to calculate their
<o:p></o:p></span></p>
<p class="MsoNormal" style="text-indent:36.0pt"><span style="font-size:11.0pt;color:#212121">knowledge of the topic and inversely adapt their response to that (be more cautious, not provide citations,
<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:11.0pt;color:#212121">warn users their knowledge is weak, etc). *THAT* would make me somewhat hopeful ...<br>
<br>
Secondly, "but then I found" fake citations -- these are a *REALLY* serious issue, especially to very old
<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:11.0pt;color:#212121">sources, e.g. the 100+ years old Irish Times citations I'd to check -- it took a full 2 days of my time plus an
<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:11.0pt;color:#212121">archivist to check them, and he spent the previous day physically retrieving the papers from a remote
<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:11.0pt;color:#212121">archive (and presumably another day returning them) -- I got this effort gratis because of the novelty of
<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:11.0pt;color:#212121">our investigation at that time, but it'd be unlikely to be repeated now without funding to pay for staff effort.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:11.0pt;color:#212121">Imagine some years from now when huge numbers of these fake citations have to be checked, simply<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:11.0pt;color:#212121">because one might be a previously unknown real citation. Who on earth is going to fund that? <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#212121"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#212121">Cheers<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#212121"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#212121">Brian Randell</span><span style="font-size:12.0pt;mso-fareast-language:EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">-- <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Helvetica;color:black">School of Computing, Newcastle University, 1 Science Square, Newcastle upon Tyne, NE4 5TG</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Helvetica;color:black">EMAIL =
<a href="mailto:Brian.Randell@ncl.ac.uk"><span style="color:#467886">Brian.Randell@ncl.ac.uk</span></a> PHONE = +44 (0)786 7805578</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Helvetica;color:black">URL =
<a href="https://www.ncl.ac.uk/computing/staff/profile/brianrandell.html"><span style="color:#467886">https://www.ncl.ac.uk/computing/staff/profile/brianrandell.html</span></a></span><span style="font-size:12.0pt"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div id="mail-editor-reference-message-container">
<div>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:12.0pt;margin-left:36.0pt">
<b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Brian Randell <brian.randell@newcastle.ac.uk><br>
<b>Date: </b>Wednesday, 7 May 2025 at 14:52<br>
<b>To: </b>Adam Hyland <achyland@uw.edu><br>
<b>Cc: </b>Sigcis <members@sigcis.org><br>
<b>Subject: </b>Re: [SIGCIS-Members] Perplexity<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt">Hi Adam:</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt">Thanks – I hadn’t realised that URL links to Perplexity’s answers aren’t transferable.</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt">It is certainly the case that Perplexity (which is based on ChatGPT4 I believe) is much more “knowledgeable” about Percy Ludgate than ChatGPT was – not least because it has ingested
subsequent additions to the Internet. </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt">Indeed, given the huge amount of effort and money being poured into LLM research we surely should expect the latest models to be significantly improved.
</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt">And Perplexity’s answer to my question “<span style="color:black">Are the references that Perplexity provides always real ones” was very reasonable, one might even say “thoughtful”,
though its cautionary comments have of course to be applied to this answer itself – a nice example of recursion.
</span></span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt;color:black">However, the NY Times had a very interesting, and surely worrying (for the Artificial Intelligentsia, especially) piece yesterday entitled: “A.I. Is Getting More Powerful,
but Its Hallucinations Are Getting Worse: </span><span style="font-size:12.0pt;color:#121212;background:white">A new wave of “reasoning” systems from companies like OpenAI is producing incorrect information more often. Even the companies don’t know why.</span><span style="font-size:12.0pt;color:black">”</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt;color:black">I hope this link to it will work:
</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt"><a href="https://www.nytimes.com/2025/05/05/technology/ai-hallucinations-chatgpt-google.html?unlocked_article_code=1.E08.mKzL.qenddVF5UQVq&smid=url-share" title="https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.nytimes.com%2F2025%2F05%2F05%2Ftechnology%2Fai-hallucinations-chatgpt-google.html%3Funlocked_article_code%3D1.E08.mKzL.qenddVF5UQVq%26smid%3Durl-share&data=05%7C02%7Cbrian.randell%40new"><span style="color:#96607D">https://www.nytimes.com/2025/05/05/technology/ai-hallucinations-chatgpt-google.html?unlocked_article_code=1.E08.mKzL.qenddVF5UQVq&smid=url-share</span></a><span style="color:black">
</span></span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt;color:black">And on the subject of “hallucinations” (or rather “fabrications”), I also strongly recommend Gary Marcus’s very recent piece:</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt;color:#212121"> <a href="https://garymarcus.substack.com/p/why-do-large-language-models-hallucinate" title="https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgarymarcus.substack.com%2Fp%2Fwhy-do-large-language-models-hallucinate&data=05%7C02%7Cbrian.randell%40newcastle.ac.uk%7C31991d59bd3a47c76dbd08dd8c8da9fc%7C9c5012c9b61644c2a91766814fbe3e87%7"><span style="color:#0078D7">Why
DO large language models hallucinate?</span></a> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt;color:#212121">(<a href="https://garymarcus.substack.com/p/why-do-large-language-models-hallucinate" title="https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgarymarcus.substack.com%2Fp%2Fwhy-do-large-language-models-hallucinate&data=05%7C02%7Cbrian.randell%40newcastle.ac.uk%7C31991d59bd3a47c76dbd08dd8c8da9fc%7C9c5012c9b61644c2a91766814fbe3e87%7"><span style="color:#96607D">https://garymarcus.substack.com/p/why-do-large-language-models-hallucinate</span></a>)</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt;color:black">Let me end by quoting from it:</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt;color:black">“</span><span style="font-size:12.0pt;color:#363737">Because LLMS statistically mimic the language people have used, they often <i>fool people </i>into thinking that
they operate like people. But they don’t operate like people. They don’t, for example, ever <i>fact check</i> (as humans sometimes, when well motivated, do). They <i>mimic the kinds of things of people say in various contexts</i>. And that’s essentially all
they do. . .</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt;line-height:19.2pt"><span style="font-size:12.0pt;color:#363737">Of course the systems are probabilistic; not every LLM will produce a hallucination every time. But the problem is not going away; OpenAI’s recent
o3 <a href="https://www.theleftshift.com/openai-admits-newer-models-hallucinate-even-more/">actually hallucinates<span style="color:windowtext;text-decoration:none"> </span></a><i><a href="https://www.theleftshift.com/openai-admits-newer-models-hallucinate-even-more/">more</a></i><a href="https://www.theleftshift.com/openai-admits-newer-models-hallucinate-even-more/"><span style="color:windowtext;text-decoration:none"> </span>than
some its predecesssors</a>. </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt;line-height:19.2pt"><span style="font-size:12.0pt;color:#363737">The chronic problem with creating <a href="https://www.forbes.com/sites/larsdaniel/2025/01/29/the-irony-ai-experts-testimony--collapses-over-fake-ai-citations/">fake
citations in research papers</a> and <a href="https://www.rawstory.com/lindell-dominion-2671843170/">faked cases in legal briefs</a> is a manifestation of the same problem; LLMs correctly “model” the structure of academic references, but often make up titles,
page numbers, journals and so on — once again failing to sanity check their outputs against information (in this case lists of references) that are readily found on the internet. So to is the<a href="https://www.vals.ai/benchmarks/finance_agent-04-22-2025"><span style="color:windowtext;text-decoration:none"> </span>rampant
problem with numerical errors in financial reports</a>, documented in a recent benchmark.</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt;line-height:19.2pt"><span style="font-size:12.0pt;color:#363737">Just how bad is it? One recent study showed <a href="https://research.aimultiple.com/ai-hallucination/">rates of hallucinations of between 15% and
60%<span style="color:windowtext;text-decoration:none"> </span></a>across various models on a benchmark of 60 questions that were <i>easily verifiable relative to easily found CNN source articles that were directly supplied in the exam</i>. Even the best performance
(15% hallucination rate) is, relative to an open-book exam with sources supplied, pathetic. That same study reports that, “According to Deloitte, 77% of businesses who joined the study are concerned about AI hallucinations”.</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt;line-height:19.2pt"><span style="font-size:12.0pt;color:#363737">If I can be blunt, it is an absolute embarrassment that a technology that has collectively cost about half a trillion dollars can’t do something
as basic as (reliably) check its output against wikipedia or a CNN article that is handed on a silver plattter. But LLMs still cannot - and on their own may never be able to — reliably do even things that basic.”</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt;color:black">Cheers</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt;color:black">Brian</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt">-- </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:9.0pt;font-family:Helvetica;color:black">School of Computing, Newcastle University, 1 Science Square, Newcastle upon Tyne, NE4 5TG</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:9.0pt;font-family:Helvetica;color:black">EMAIL =
<a href="mailto:Brian.Randell@ncl.ac.uk"><span style="color:#467886">Brian.Randell@ncl.ac.uk</span></a> PHONE = +44 (0)786 7805578</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:9.0pt;font-family:Helvetica;color:black">URL =
<a href="https://www.ncl.ac.uk/computing/staff/profile/brianrandell.html"><span style="color:#467886">https://www.ncl.ac.uk/computing/staff/profile/brianrandell.html</span></a></span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<div id="mail-editor-reference-message-container">
<div>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:12.0pt;margin-left:72.0pt">
<b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Adam Hyland <achyland@uw.edu><br>
<b>Date: </b>Tuesday, 6 May 2025 at 23:15<br>
<b>To: </b>Brian Randell <brian.randell@newcastle.ac.uk><br>
<b>Cc: </b>Sigcis <members@sigcis.org><br>
<b>Subject: </b>Re: [SIGCIS-Members] Perplexity</span><o:p></o:p></p>
</div>
<div style="border:none;border-left:solid #E6E68A 4.5pt;padding:0cm 0cm 0cm 0cm;margin-bottom:3.75pt">
<p style="mso-margin-top-alt:5.0pt;margin-right:0cm;margin-bottom:0cm;margin-left:72.0pt;background:#FFFFCC">
<span style="font-size:11.0pt;font-family:"Apple Color Emoji";color:black">⚠</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black"> External sender. Take care when opening links or attachments. Do not provide your login details.
</span><o:p></o:p></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt">Thanks for your return to this topic. The perplexity thread you shared is private and unfortunately the pdf generated obscures the answer with a UI element. I’ve asked the same question
to Perplexity using their “research” mode which more profligately uses computing resources to give a more detailed answer. The result is here (</span><o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"><a href="https://www.perplexity.ai/search/did-percy-ludgate-s-work-have-oxneW37nS9.bAiYvfV3QVw">https://www.perplexity.ai/search/did-percy-ludgate-s-work-have-oxneW37nS9.bAiYvfV3QVw</a>)
and should be available to anyone with the URL.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt">I’m familiar with perplexity because the University of Washington pays for “pro” access which makes it both partially difficult and morally dubious to proscribe its use in the classroom.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt">One broad point is probably obvious to everyone but bears repeating: there is much more difference in capability between any of these agents today versus a year ago than there is
among the agents themselves. They all show a dramatic increase in capability to “understand” queries and generate cogent responses. If someone on this thread last posed one of their exam questions to ChatGPT last year they ought to try again today, and again
in another month.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt">-Adam</span><o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"><br clear="all">
</span><o:p></o:p></p>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt">Adam Hyland (<i>he/him)</i>
</span><o:p></o:p></p>
<div>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"><a href="https://adampunk.com/" target="_blank">adampunk.com</a></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt">UW HCDE PhD Student</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt">On Tue, May 6, 2025 at 8:57</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif"> </span><span style="font-size:12.0pt">AM Brian Randell via Members <<a href="mailto:members@lists.sigcis.org" target="_blank">members@lists.sigcis.org</a>>
wrote:</span><o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt">Hi:</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt">A while ago I and colleagues published a short critique in the Annals of the History of Computing of ChatGPT, based on its performance on some questions about Percy Ludgate.</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt">I’ve recently been trying (the free version of) Perplexity, the AI search (or more exactly question-answering) system.</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt">Perplexity itself claims:</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt;text-indent:36.0pt"><span style="font-size:12.0pt">Here's what makes Perplexity different</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt;text-indent:36.0pt"><span style="font-size:12.0pt">Answers that are accurate and always cited</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt;text-indent:36.0pt"><span style="font-size:12.0pt">We continuously search the internet and identify the best sources, from academic research
</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt;text-indent:36.0pt"><span style="font-size:12.0pt">to Reddit threads, to provide the perfect answer to any question.</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt;text-indent:36.0pt"><span style="font-size:12.0pt">Citations in every response</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt;text-indent:36.0pt"><span style="font-size:12.0pt">Every answer uses cited sources to provide a more accurate and comprehensive answer.
</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt;text-indent:36.0pt"><span style="font-size:12.0pt">If you want to dig deeper, just click the link to the source.</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt">See its brilliant answer to the question “</span><span style="font-size:13.5pt;font-family:"Segoe UI",sans-serif">Did Percy Ludgate's work have any impact?”:</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"><a href="https://urldefense.com/v3/__https://www.perplexity.ai/search/did-percy-ludgate-s-work-have-2esemV.BRuCo0Q7O_4AxfA__;!!K-Hz7m0Vt54!gyxp0x4-UgdlqkBkNCRifXnXGAv6Y1rWzqMDvgDAx9lcE0e3xttiUQD_Zgg0jayzgaPV_LCgiwp1l7QbFBQe$" target="_blank">https://www.perplexity.ai/search/did-percy-ludgate-s-work-have-2esemV.BRuCo0Q7O_4AxfA</a></span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt">The web version limits the number of questions per day – so far the iPhone App hasn’t.</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt">I assume I’m not alone here in trying Perplexity, but I don’t recall any previous comment about it in SIGCIS.</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt">However, the Wikipedia article about it
</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"> <a href="https://urldefense.com/v3/__https://en.wikipedia.org/wiki/Perplexity_AI__;!!K-Hz7m0Vt54!gyxp0x4-UgdlqkBkNCRifXnXGAv6Y1rWzqMDvgDAx9lcE0e3xttiUQD_Zgg0jayzgaPV_LCgiwp1l_ByERqJ$" target="_blank">https://en.wikipedia.org/wiki/Perplexity_AI</a></span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt">is a little sobering, regarding its alleged copyright violations, and failure to respect the robots.txt web-crawling standards.</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt">Cheers</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt">Brian</span><o:p></o:p></p>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-left:72.0pt"><span style="font-size:12.0pt">_______________________________________________<br>
This email is relayed from members at <a href="http://sigcis.org/" target="_blank">
sigcis.org</a>, the email discussion list of SHOT SIGCIS. Opinions expressed here are those of the member posting and are not reviewed, edited, or endorsed by SIGCIS. The list archives are at
<a href="https://urldefense.com/v3/__http://lists.sigcis.org/pipermail/members-sigcis.org/__;!!K-Hz7m0Vt54!gyxp0x4-UgdlqkBkNCRifXnXGAv6Y1rWzqMDvgDAx9lcE0e3xttiUQD_Zgg0jayzgaPV_LCgiwp1lxruqEdl$" target="_blank">
https://urldefense.com/v3/__http://lists.sigcis.org/pipermail/members-sigcis.org/__;!!K-Hz7m0Vt54!gyxp0x4-UgdlqkBkNCRifXnXGAv6Y1rWzqMDvgDAx9lcE0e3xttiUQD_Zgg0jayzgaPV_LCgiwp1lxruqEdl$</a> and you can change your subscription options at
<a href="https://urldefense.com/v3/__http://lists.sigcis.org/listinfo.cgi/members-sigcis.org__;!!K-Hz7m0Vt54!gyxp0x4-UgdlqkBkNCRifXnXGAv6Y1rWzqMDvgDAx9lcE0e3xttiUQD_Zgg0jayzgaPV_LCgiwp1lwGqTJnR$" target="_blank">
https://urldefense.com/v3/__http://lists.sigcis.org/listinfo.cgi/members-sigcis.org__;!!K-Hz7m0Vt54!gyxp0x4-UgdlqkBkNCRifXnXGAv6Y1rWzqMDvgDAx9lcE0e3xttiUQD_Zgg0jayzgaPV_LCgiwp1lwGqTJnR$</a>
</span><o:p></o:p></p>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>