<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
Some thoughts on The Book from the wonderful Carl Sagan seem apposite:
<div>
<h1 class="quoteText"><font size="4" style="font-weight: normal;">“What an astonishing thing a book is. It's a flat object made from a tree with flexible parts on which are imprinted lots of funny dark squiggles. But one glance at it and you're inside the mind
of another person, maybe somebody dead for thousands of years. Across the millennia, an author is speaking clearly and silently inside your head, directly to you. Writing is perhaps the greatest of human inventions, binding together people who never knew each
other, citizens of distant epochs. Books break the shackles of time. A book is proof that humans are capable of working magic."</font></h1>
<div>Dag</div>
<div>
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div dir="auto" style="text-align: start; text-indent: 0px; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div dir="auto" style="text-align: start; text-indent: 0px; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div dir="auto" style="text-align: start; text-indent: 0px; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div dir="auto" style="text-align: start; text-indent: 0px; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div dir="auto" style="text-align: start; text-indent: 0px; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div dir="auto" style="text-align: start; text-indent: 0px; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div dir="auto" style="text-align: start; text-indent: 0px; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div dir="auto" style="text-align: start; text-indent: 0px; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div dir="auto" style="text-align: start; text-indent: 0px; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div dir="auto" style="text-align: start; text-indent: 0px; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; -webkit-text-stroke-width: 0px;">
-----<br>
Dag Spicer<br>
Senior Curator<br>
Computer History Museum<br>
Editorial Board, IEEE Annals of the History of Computing<br>
ACM History Committee<br>
1401 N. Shoreline Blvd.<br>
Mountain View CA 94043</div>
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; -webkit-text-stroke-width: 0px;">
<br>
</div>
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; -webkit-text-stroke-width: 0px;">
“History is a vast early warning system.” </div>
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; -webkit-text-stroke-width: 0px;">
— Norman Cousins, American journalist (1915-1990).</div>
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; -webkit-text-stroke-width: 0px;">
<br>
</div>
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; -webkit-text-stroke-width: 0px;">
Join our Mailing List here: <a href="https://info.computerhistory.org/subscribe">https://info.computerhistory.org/subscribe</a> </div>
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; -webkit-text-stroke-width: 0px;">
<br>
</div>
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; -webkit-text-stroke-width: 0px;">
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div><br>
<blockquote type="cite">
<div>On May 24, 2025, at 2:01 AM, Willard McCarty via Members <members@lists.sigcis.org> wrote:</div>
<br class="Apple-interchange-newline">
<div>
<div><br>
<br>
An intrusion from an outsider who subscribes...<br>
<br>
This morning on BBC Radio 4 someone at the Hay-on-Wye Festival<br>
(celebrating books) got to talking about the pleasure of reading<br>
physical codices. That led me to thinking about the inevitable loss of<br>
data (if you will) when a book, any book, is digitised. I couldn't do<br>
the interdisciplinary work I do without access to a large number of<br>
digitised texts. Indeed I am exceedingly grateful for digital<br>
collections and build my own with the help of a good scanner + OCR. But<br>
then there's the sustained reading of those I regard as important to<br>
spend time with.<br>
<br>
However such an argument runs, the fact is that digitisation is a<br>
particularly lossy form of translation. More than that, it not so much<br>
makes the scholar's life easier as quite different. I worry about the<br>
'easier' bit because it entails loss that one quickly stops thinking<br>
about--hence the tendency to miss what only hard work will find. Given<br>
the competitiveness of the academic life esp. for junior scholars--I'd<br>
guess that in this crowd I don't have to spell it out.<br>
<br>
On the tangible/digital, not either/or but both!<br>
<br>
All best,<br>
Willard McCarty<br>
<br>
On 23/05/2025 22:39, James Cortada via Members wrote:<br>
<blockquote type="cite">To your second point, what if the old Irish newspapers were no longer<br>
around to check? We have 90 years of experience of librarians filming<br>
and digitizing newspapers, magazines, etc then throwing away the<br>
originals, at least in the USA. That was especially so with American<br>
librarians with newspapers beginning in the 1930s when they microfilmed<br>
them. Hate to criticize them, but they sometimes did not film all the<br>
issues of a newspaper, sometimes missed pages, or they came out blurry.<br>
Fast forward to the future if AI invents citations to stuff that no<br>
longer exists in their original. I encountered a bit of that similar<br>
result with the Google book project a few years ago. I worry about this<br>
possibility. How do we address that problem?<br>
<br>
Meanwhile, I, an old guy who grew up pre-AI, will still look at hard<br>
copies or trusted online versions of something (e.g., a journal) and<br>
keep hard copies of anything I cite off the Internet (e.g., images)<br>
before dropping that info into an endnote, because I don't want to be<br>
accused of faking stuff. It's hard enough working with real facts and<br>
sources. I suggest others follow a similar path. Call me old<br>
fashioned, but it has not hurt my productivity so far. Thanks for<br>
posting your thoughts, Brian. Jim Cortada<br>
<br>
On Fri, May 23, 2025 at 4:22 PM Brian Randell via Members<br>
<members@lists.sigcis.org <mailto:members@lists.sigcis.org>> wrote:<br>
<br>
Hi:____<br>
<br>
__ __<br>
<br>
My colleague Brian Coghlan added two further points about the<br>
dangers AI hallucinations/fabrications pose to historians who are<br>
trying to use Generative AI tools to help their investigations:____<br>
<br>
__ __<br>
<br>
First, our own investigation very strongly implied the "AI error<br>
rate" was inversely proportional to the AI ____<br>
<br>
knowledge of the query topic. So one *very easy* countermeasure<br>
would be for AI tools to calculate their ____<br>
<br>
knowledge of the topic and inversely adapt their response to that<br>
(be more cautious, not provide citations, ____<br>
<br>
warn users their knowledge is weak, etc). *THAT* would make me<br>
somewhat hopeful ...<br>
<br>
Secondly, "but then I found" fake citations -- these are a *REALLY*<br>
serious issue, especially to very old ____<br>
<br>
sources, e.g. the 100+ years old Irish Times citations I'd to check<br>
-- it took a full 2 days of my time plus an ____<br>
<br>
archivist to check them, and he spent the previous day physically<br>
retrieving the papers from a remote ____<br>
<br>
archive (and presumably another day returning them) -- I got this<br>
effort gratis because of the novelty of ____<br>
<br>
our investigation at that time, but it'd be unlikely to be repeated<br>
now without funding to pay for staff effort.____<br>
<br>
Imagine some years from now when huge numbers of these fake<br>
citations have to be checked, simply____<br>
<br>
because one might be a previously unknown real citation. Who on<br>
earth is going to fund that? ____<br>
<br>
__ __<br>
<br>
Cheers____<br>
<br>
__ __<br>
<br>
Brian Randell____<br>
<br>
__ __<br>
<br>
-- ____<br>
<br>
__ __<br>
<br>
School of Computing, Newcastle University, 1 Science Square,<br>
Newcastle upon Tyne, NE4 5TG____<br>
<br>
EMAIL = Brian.Randell@ncl.ac.uk <mailto:Brian.Randell@ncl.ac.uk><br>
PHONE = +44 (0)786 7805578____<br>
<br>
URL = https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ncl.ac.uk%2Fcomputing%2Fstaff%2Fprofile%2F&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546393139%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=oNnyTHMuMgMucXs1HQaF7fFQ2aPrePKtCUKz%2FgOWL3I%3D&reserved=0<br>
brianrandell.html <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ncl.ac.uk%2Fcomputing%2Fstaff%2Fprofile%2F&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546414984%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=fIctrwGi4ixQuD4q3NTvxNmvkkTHWp7QVNz%2FUgsYM7w%3D&reserved=0<br>
brianrandell.html>____<br>
<br>
__ __<br>
<br>
__ __<br>
<br>
__ __<br>
<br>
*From: *Brian Randell <brian.randell@newcastle.ac.uk<br>
<mailto:brian.randell@newcastle.ac.uk>><br>
*Date: *Wednesday, 7 May 2025 at 14:52<br>
*To: *Adam Hyland <achyland@uw.edu <mailto:achyland@uw.edu>><br>
*Cc: *Sigcis <members@sigcis.org <mailto:members@sigcis.org>><br>
*Subject: *Re: [SIGCIS-Members] Perplexity____<br>
<br>
Hi Adam:____<br>
<br>
____<br>
<br>
Thanks – I hadn’t realised that URL links to Perplexity’s answers<br>
aren’t transferable.____<br>
<br>
____<br>
<br>
It is certainly the case that Perplexity (which is based on ChatGPT4<br>
I believe) is much more “knowledgeable” about Percy Ludgate than<br>
ChatGPT was – not least because it has ingested subsequent additions<br>
to the Internet. ____<br>
<br>
____<br>
<br>
Indeed, given the huge amount of effort and money being poured into<br>
LLM research we surely should expect the latest models to be<br>
significantly improved. ____<br>
<br>
____<br>
<br>
And Perplexity’s answer to my question “Are the references that<br>
Perplexity provides always real ones” was very reasonable, one might<br>
even say “thoughtful”, though its cautionary comments have of course<br>
to be applied to this answer itself – a nice example of recursion. ____<br>
<br>
____<br>
<br>
However, the NY Times had a very interesting, and surely worrying<br>
(for the Artificial Intelligentsia, especially) piece yesterday<br>
entitled: “A.I. Is Getting More Powerful, but Its Hallucinations Are<br>
Getting Worse: A new wave of “reasoning” systems from companies like<br>
OpenAI is producing incorrect information more often. Even the<br>
companies don’t know why.”____<br>
<br>
____<br>
<br>
I hope this link to it will work: ____<br>
<br>
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.nytimes.com%2F2025%2F05%2F05%2Ftechnology%2Fai-hallucinations-&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546436545%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=gEVbAtuBLymTKWaFwz9h%2Be6vYbq5q2UNlVdPUpDvRE0%3D&reserved=0<br>
chatgpt-google.html?<br>
unlocked_article_code=1.E08.mKzL.qenddVF5UQVq&smid=url-share<br>
<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.nytimes.com%2F2025%2F05%2F05%2Ftechnology%2Fai-hallucinations-&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546455331%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=vRVlqD9IhBHHxMDZHD6gukKl2tzL%2B7HvgQ1MdNMoV7U%3D&reserved=0<br>
chatgpt-google.html?<br>
unlocked_article_code=1.E08.mKzL.qenddVF5UQVq&smid=url-share>____<br>
<br>
____<br>
<br>
And on the subject of “hallucinations” (or rather “fabrications”), I<br>
also strongly recommend Gary Marcus’s very recent piece:____<br>
<br>
____<br>
<br>
Why DO large language models hallucinate? <https://<br>
garymarcus.substack.com/p/why-do-large-language-models-hallucinate> ____<br>
<br>
(https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgarymarcus.substack.com%2Fp%2Fwhy-do-large-language-models-&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546472543%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=xPJ2YSxEUEEBw9YRCf7qod1tjpyjd52GxXOWNmzKYdU%3D&reserved=0<br>
hallucinate <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgarymarcus.substack.com%2Fp%2Fwhy-do-large-&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546489849%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=cyClLhEZ5tXUcNiWnOY0%2BTZUQI2OvxYAFj24rnGi3hY%3D&reserved=0<br>
language-models-hallucinate>)____<br>
<br>
____<br>
<br>
Let me end by quoting from it:____<br>
<br>
____<br>
<br>
“Because LLMS statistically mimic the language people have used,<br>
they often /fool people /into thinking that they operate like<br>
people. But they don’t operate like people. They don’t, for example,<br>
ever /fact check/ (as humans sometimes, when well motivated, do).<br>
They /mimic the kinds of things of people say in various contexts/.<br>
And that’s essentially all they do. . .____<br>
<br>
Of course the systems are probabilistic; not every LLM will produce<br>
a hallucination every time. But the problem is not going away;<br>
OpenAI’s recent o3 actually hallucinates<https://<br>
https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.theleftshift.com%2Fopenai-admits-newer-models-hallucinate-even-&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546506529%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=xw5N%2Fc6tdtVnuVi4D03pyaPlcthOWxyKWT02P5Ofjss%3D&reserved=0<br>
more/>/more <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.theleftshift.com%2Fopenai-admits-newer-&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546521577%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=bQjrSC%2B7xv8IYTFYlHcrdXFCDt8h9eZuoVhYJ6GqH1U%3D&reserved=0<br>
models-hallucinate-even-more/>/than some its predecesssors <https://<br>
https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.theleftshift.com%2Fopenai-admits-newer-models-hallucinate-even-&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546534705%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=ZLpsZutayii5WFyyxofOWiVMASWx1O0nu4picq8gVdc%3D&reserved=0<br>
more/>. ____<br>
<br>
The chronic problem with creating fake citations in research papers<br>
<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.forbes.com%2Fsites%2Flarsdaniel%2F2025%2F01%2F29%2Fthe-irony-ai-&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546547635%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=HD1jw8wsOJghYC6LIxA6MpL%2FI7YU9HEy69mugPUys8o%3D&reserved=0<br>
experts-testimony--collapses-over-fake-ai-citations/> and faked<br>
cases in legal briefs <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.rawstory.com%2Flindell-&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546560792%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=gyX8uT%2Fug31s5K8zkvG1VujsP%2FX8ZOnsa%2Fdz85tECU0%3D&reserved=0<br>
dominion-2671843170/> is a manifestation of the same problem; LLMs<br>
correctly “model” the structure of academic references, but often<br>
make up titles, page numbers, journals and so on — once again<br>
failing to sanity check their outputs against information (in this<br>
case lists of references) that are readily found on the internet. So<br>
to is therampant problem with numerical errors in financial reports<br>
<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.vals.ai%2Fbenchmarks%2Ffinance_agent-04-22-2025&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546576681%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Jy66QrskB3aPCoO3ayT%2FICWZ4FlbqZq5M6SlRgAbyZM%3D&reserved=0>,<br>
documented in a recent benchmark.____<br>
<br>
Just how bad is it? One recent study showed rates of hallucinations<br>
of between 15% and 60%<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fresearch.aimultiple.com%2Fai-&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546590578%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=seRW1vLe2NuBP15YS%2F%2FmAtHGOCE%2B5838OzedZyVUnN0%3D&reserved=0<br>
hallucination/>across various models on a benchmark of 60 questions<br>
that were /easily verifiable relative to easily found CNN source<br>
articles that were directly supplied in the exam/. Even the best<br>
performance (15% hallucination rate) is, relative to an open-book<br>
exam with sources supplied, pathetic. That same study reports that,<br>
“According to Deloitte, 77% of businesses who joined the study are<br>
concerned about AI hallucinations”.____<br>
<br>
If I can be blunt, it is an absolute embarrassment that a technology<br>
that has collectively cost about half a trillion dollars can’t do<br>
something as basic as (reliably) check its output against wikipedia<br>
or a CNN article that is handed on a silver plattter. But LLMs still<br>
cannot - and on their own may never be able to — reliably do even<br>
things that basic.”____<br>
<br>
Cheers____<br>
<br>
____<br>
<br>
Brian____<br>
<br>
____<br>
<br>
-- ____<br>
<br>
____<br>
<br>
School of Computing, Newcastle University, 1 Science Square,<br>
Newcastle upon Tyne, NE4 5TG____<br>
<br>
EMAIL = Brian.Randell@ncl.ac.uk <mailto:Brian.Randell@ncl.ac.uk><br>
PHONE = +44 (0)786 7805578____<br>
<br>
URL = https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ncl.ac.uk%2Fcomputing%2Fstaff%2Fprofile%2F&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546603193%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=xVmA%2FdBbDaP81326OABeW%2FbePJz4Z8aihr%2F3dSqt7zE%3D&reserved=0<br>
brianrandell.html <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ncl.ac.uk%2Fcomputing%2Fstaff%2Fprofile%2F&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546615899%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=QnN96y8EPhon8PY3WVdoggoWF0o3j23NTDpPYSGZlBk%3D&reserved=0<br>
brianrandell.html>____<br>
<br>
____<br>
<br>
____<br>
<br>
____<br>
<br>
*From: *Adam Hyland <achyland@uw.edu <mailto:achyland@uw.edu>><br>
*Date: *Tuesday, 6 May 2025 at 23:15<br>
*To: *Brian Randell <brian.randell@newcastle.ac.uk<br>
<mailto:brian.randell@newcastle.ac.uk>><br>
*Cc: *Sigcis <members@sigcis.org <mailto:members@sigcis.org>><br>
*Subject: *Re: [SIGCIS-Members] Perplexity____<br>
<br>
⚠External sender. Take care when opening links or attachments. Do<br>
not provide your login details. ____<br>
<br>
Thanks for your return to this topic. The perplexity thread you<br>
shared is private and unfortunately the pdf generated obscures the<br>
answer with a UI element. I’ve asked the same question to Perplexity<br>
using their “research” mode which more profligately uses computing<br>
resources to give a more detailed answer. The result is here (____<br>
<br>
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.perplexity.ai%2Fsearch%2Fdid-percy-ludgate-s-work-have-&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546629150%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=8QIVbhLoazvcEGTCFS3nRUmDn2neKBJFd7nq8EkVJ1o%3D&reserved=0<br>
oxneW37nS9.bAiYvfV3QVw <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.perplexity.ai%2Fsearch%2Fdid-percy-&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546642455%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=0yHeqZbyjt5RHEE6eq%2FZoJ0J1UcyJfR9tBSJQSSQ4sA%3D&reserved=0<br>
ludgate-s-work-have-oxneW37nS9.bAiYvfV3QVw>) and should be available<br>
to anyone with the URL.____<br>
<br>
____<br>
<br>
I’m familiar with perplexity because the University of Washington<br>
pays for “pro” access which makes it both partially difficult and<br>
morally dubious to proscribe its use in the classroom.____<br>
<br>
____<br>
<br>
One broad point is probably obvious to everyone but bears repeating:<br>
there is much more difference in capability between any of these<br>
agents today versus a year ago than there is among the agents<br>
themselves. They all show a dramatic increase in capability to<br>
“understand” queries and generate cogent responses. If someone on<br>
this thread last posed one of their exam questions to ChatGPT last<br>
year they ought to try again today, and again in another month.____<br>
<br>
____<br>
<br>
-Adam____<br>
<br>
<br>
____<br>
<br>
Adam Hyland (/he/him)/ ____<br>
<br>
adampunk.com <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fadampunk.com%2F&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546655285%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Ow8%2FV0Y7rWMGRQhane6Pp%2FcE9Oxg55j0zNPTYgT8bBg%3D&reserved=0>____<br>
<br>
UW HCDE PhD Student____<br>
<br>
____<br>
<br>
____<br>
<br>
On Tue, May 6, 2025 at 8:57 AM Brian Randell via Members<br>
<members@lists.sigcis.org <mailto:members@lists.sigcis.org>> wrote:____<br>
<br>
Hi:____<br>
<br>
____<br>
<br>
A while ago I and colleagues published a short critique in the<br>
Annals of the History of Computing of ChatGPT, based on its<br>
performance on some questions about Percy Ludgate.____<br>
<br>
____<br>
<br>
I’ve recently been trying (the free version of) Perplexity, the<br>
AI search (or more exactly question-answering) system.____<br>
<br>
____<br>
<br>
Perplexity itself claims:____<br>
<br>
____<br>
<br>
Here's what makes Perplexity different____<br>
<br>
____<br>
<br>
Answers that are accurate and always cited____<br>
<br>
We continuously search the internet and identify the best<br>
sources, from academic research ____<br>
<br>
to Reddit threads, to provide the perfect answer to any<br>
question.____<br>
<br>
____<br>
<br>
Citations in every response____<br>
<br>
Every answer uses cited sources to provide a more accurate and<br>
comprehensive answer. ____<br>
<br>
If you want to dig deeper, just click the link to the source.____<br>
<br>
____<br>
<br>
See its brilliant answer to the question “Did Percy Ludgate's<br>
work have any impact?”:____<br>
<br>
____<br>
<br>
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.perplexity.ai%2Fsearch%2Fdid-percy-ludgate-s-work-&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546668082%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=dzyBnIvPfmq65vIvQjJPwvPZHANmdBEFbwA%2FIIPaVdw%3D&reserved=0<br>
have-2esemV.BRuCo0Q7O_4AxfA <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546681405%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=oMVxUq3W6zK%2BIoz3VDH1V5sIJ8LsSD8KEo1aSvqOtm0%3D&reserved=0<br>
__https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.perplexity.ai%2Fsearch%2Fdid-percy-ludgate-s-work-&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546694395%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=nItNM3SOH4x4wTqFMVD56VVdYHMtgQiK8RCJ8GBL3i8%3D&reserved=0<br>
have-2esemV.BRuCo0Q7O_4AxfA__;!!K-Hz7m0Vt54!gyxp0x4-<br>
UgdlqkBkNCRifXnXGAv6Y1rWzqMDvgDAx9lcE0e3xttiUQD_Zgg0jayzgaPV_LCgiwp1l7QbFBQe$>____<br>
<br>
____<br>
<br>
The web version limits the number of questions per day – so far<br>
the iPhone App hasn’t.____<br>
<br>
____<br>
<br>
I assume I’m not alone here in trying Perplexity, but I don’t<br>
recall any previous comment about it in SIGCIS.____<br>
<br>
____<br>
<br>
However, the Wikipedia article about it ____<br>
<br>
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FPerplexity_AI&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546706849%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=ttz0ePB2g%2BxEWWIKeQcWtVWDB1FkyzqTPxuJo%2BQuZPE%3D&reserved=0
<https://<br>
urldefense.com/v3/__https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2F&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546719376%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=0PAz3wNJkFpfXf%2BrFq5X2ndkK9Ag1geNAVatxGlAwgc%3D&reserved=0<br>
Perplexity_AI__;!!K-Hz7m0Vt54!gyxp0x4-<br>
UgdlqkBkNCRifXnXGAv6Y1rWzqMDvgDAx9lcE0e3xttiUQD_Zgg0jayzgaPV_LCgiwp1l_ByERqJ$>____<br>
<br>
is a little sobering, regarding its alleged copyright<br>
violations, and failure to respect the robots.txt web-crawling<br>
standards.____<br>
<br>
____<br>
<br>
Cheers____<br>
<br>
____<br>
<br>
Brian____<br>
<br>
_______________________________________________<br>
This email is relayed from members at sigcis.org <http://<br>
sigcis.org/>, the email discussion list of SHOT SIGCIS. Opinions<br>
expressed here are those of the member posting and are not<br>
reviewed, edited, or endorsed by SIGCIS. The list archives are<br>
at https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__http%3A%2F%2Flists.sigcis.org%2F&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546734328%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=mNTHHgtSvVSBbyjJnHzo8v9L5h35RTOFbE1wt3YK8Hg%3D&reserved=0<br>
pipermail/members-sigcis.org/__;!!K-Hz7m0Vt54!gyxp0x4-<br>
UgdlqkBkNCRifXnXGAv6Y1rWzqMDvgDAx9lcE0e3xttiUQD_Zgg0jayzgaPV_LCgiwp1lxruqEdl$ <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__http%3A%2F%2Flists.sigcis.org%2Fpipermail%2Fmembers-sigcis.org%2F__%3B!!K-Hz7m0Vt54!gyxp0x4-UgdlqkBkNCRifXnXGAv6Y1rWzqMDvgDAx9lcE0e3xttiUQD_Zgg0jayzgaPV_LCgiwp1lxruqEdl%24&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546747965%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=f6nX7F1K3268ahsegkgnJjIzdjxAXPnNO0bhNCXmGYs%3D&reserved=0>
and you can change your subscription options at https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__http%3A%2F%2Flists.sigcis.org%2Flistinfo.cgi%2Fmembers-sigcis.org__%3B!!K-Hz7m0Vt54!gyxp0x4-UgdlqkBkNCRifXnXGAv6Y1rWzqMDvgDAx9lcE0e3xttiUQD_Zgg0jayzgaPV_LCgiwp1lwGqTJnR%24&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546760503%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=rYIHnjbbpNxLv98QQmLWlAQfCHs92vz1U17hSahAsSw%3D&reserved=0
<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__http%3A%2F%2Flists.sigcis.org%2Flistinfo.cgi%2Fmembers-sigcis.org__%3B!!K-Hz7m0Vt54!gyxp0x4-UgdlqkBkNCRifXnXGAv6Y1rWzqMDvgDAx9lcE0e3xttiUQD_Zgg0jayzgaPV_LCgiwp1lwGqTJnR%24&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546772708%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=xwE53BautyC9Vpn6F9r8qw86O24BApLGqyGucsZqqzA%3D&reserved=0>
____<br>
<br>
_______________________________________________<br>
This email is relayed from members at sigcis.org <http://<br>
sigcis.org>, the email discussion list of SHOT SIGCIS. Opinions<br>
expressed here are those of the member posting and are not reviewed,<br>
edited, or endorsed by SIGCIS. The list archives are at http://<br>
lists.sigcis.org/pipermail/members-sigcis.org/ <http://<br>
lists.sigcis.org/pipermail/members-sigcis.org/> and you can change<br>
your subscription options at https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.sigcis.org%2Flistinfo.cgi%2F&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546784738%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=XtcDc0HkYCyhnQ3mPdaIubbp5Wo0LWrGpSQFJ5%2BKdzE%3D&reserved=0<br>
members-sigcis.org <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.sigcis.org%2Flistinfo.cgi%2Fmembers-&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546798030%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=nwG2%2BG8yCUGsx0LBjttoVj9Z8%2BwvrOd9AoMgI8j2vAw%3D&reserved=0<br>
sigcis.org><br>
<br>
<br>
<br>
--<br>
James W. Cortada<br>
Senior Research Fellow<br>
Charles Babbage Institute<br>
University of Minnesota<br>
jcortada@umn.edu <mailto:jcortada@umn.edu><br>
608-274-6382<br>
<br>
_______________________________________________<br>
This email is relayed from members at sigcis.org, the email discussion list of SHOT SIGCIS. Opinions expressed here are those of the member posting and are not reviewed, edited, or endorsed by SIGCIS. The list archives are at https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.sigcis.org%2Fpipermail%2Fmembers-sigcis.org%2F&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546810380%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=7QZaDpifoQT10soobyZ%2FXcy3c7uvAWYc9j%2BOYHKFrx0%3D&reserved=0
and you can change your subscription options at https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.sigcis.org%2Flistinfo.cgi%2Fmembers-sigcis.org&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546822843%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=JfGl%2FObIRVGbcND3yPtsObIeULypic3vGckuvEjAiG4%3D&reserved=0<br>
</blockquote>
<br>
--<br>
Willard McCarty,<br>
Professor emeritus, King's College London;<br>
Editor, Humanist<br>
https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.mccarty.org.uk%2F&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546835757%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=XXJSH%2F54jEpy4Z1w9jGhQe%2FYvD0shfZbVTaR8uCr%2Bho%3D&reserved=0<br>
<br>
_______________________________________________<br>
This email is relayed from members at sigcis.org, the email discussion list of SHOT SIGCIS. Opinions expressed here are those of the member posting and are not reviewed, edited, or endorsed by SIGCIS. The list archives are at https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.sigcis.org%2Fpipermail%2Fmembers-sigcis.org%2F&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546849073%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=bLukzhtSLkVWdX9sjeHXmRCsKHF8NBGBmh0cDv7x%2FjI%3D&reserved=0
and you can change your subscription options at https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.sigcis.org%2Flistinfo.cgi%2Fmembers-sigcis.org&data=05%7C02%7Cdspicer%40computerhistory.org%7C38f01d8a834a4bf76b2608dd9aa1b495%7Cb6a9c12a29ee4c5f8f93f0d7a8e2db2f%7C0%7C0%7C638836741546862281%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=AABudduVx2EjcqxTmhQ2LSEg%2FLaLshsxoe%2BHmG0ry38%3D&reserved=0<br>
.<br>
<br>
CAUTION: External email. Beware of links & attachments.<br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</body>
</html>