I was encouraged last year to join this list to introduce my research and upcoming book about the history of the Berkeley Software Distributions. (My membership didn't kick in for a few months.) I am authoring a detailed book covering the history of BSD / Berkeley Unix. One of its releases is known as the "Greatest Software Ever Written" -- selected by DARPA to be the standardized operating system platform for the DARPA research community, it became the widely distributed software release that included the TCP/IP Internet protocols and Internet server software to the world. The book attempts to introduce the characters and events surrounding the BSD story. Some of my main goals were: - Interview the ~74 original surviving contributors of the early BSDs (before 1981). (Sadly at least three early participants have already deceased.) - Interview most of the contributors toward the 4.3BSD-Networking Release 2 and 4.4BSD-Lite2 distributions. (The replacement of proprietary software and key event in open source history.) - Interview the founders and some key participants of CSRG, 386BSD, BSD/386, NetBSD, FreeBSD, OpenBSD, DragonFly, etc. - Accurately and fairly share the stories for the key events in BSD history. - Cover the modern BSD histories. I have already done over 85 interviews and the book draft is currently around 270 pages long with over 200 citations in the bibliography (and over a 100 additional that need to add to the bibtex file). The book has 26 chapters and is around 67,000 words. I have over 100,000 words of notes in my subversion tex files that are commented out. (Quickly, for my related background: journalism degree from Western Washington Univ.; as a small book publisher, published over ten books; ran two BSD news/features websites; BSD sysadmin for only around 16 years; active participant in the BSD community, lecturing at events and providing marketing and project management.) I will quickly introduce some the methods used for researching my book: - downloaded old BSDs release tarballs (including coding up support to extract now extinct archive formats) and purchased CSRG archives of BSD releases (including some experimental releases). The BSD releases included documentation with lots of acknowledgements, source code with lots of comments, some code revision history, binaries, real-world configurations, including some personal files from the key developers left in home directories that were tar'ed up with the releases. For the oldest releases, I skimmed through all the source code and all the documentation (manual pages, howtos, master theses, etc.). For newer larger releases, I programmatically searched for significant details. (I also skimmed through other old Unix-related archives for BSD-related histories.) CSRG sets also included administration emails, licensing details, letters to sponsors, etc. - purchased the SCCS histories and used this to find the source code history for various code and documentation, etc. Note that some of the old sets had older code not found in the SCCS histories, plus some SCCS histories had gaps or were lost due to filename/directory moves. The BSD sets were used to fill in some holes and to find older code, including old SCCS or RCS metadata. For newer releases, I used CVS histories. - took notes from BSD lectures and found all known BSD history articles - built list of all contact names (including non-developers) for the earliest releases up to mid 1980, and then all contributors toward the later BSD releases, and then significant participants toward the modern day BSDs. I attempted contacting maybe hundreds of these participants via email or some social network sites. I heard back from over a 100. Generally before I contact the person, I search my now-massive digital collection of documents and then summarize what I know to help the participant recollect history from long ago. I did over 80 interviews by email, several by phone, and some in-person (at some conferences). All my interview notes are stored in separate per-person files. Often the participants name other people I should consider. Looking now, I see I have at least 454 names in my detailed list. I did interviews with inventors of many of the open source and Unix technologies in use today. - organized a file system with categories of different files for notes, massive TODO list, list of people to contact, books to purchase, etc, and the collection of downloaded or scanned files categorized by decades (plus I kept track of where the data was obtained). (I also keep a file of typos, errata, or other bugs I found in lectures, books, old articles or documentation, such as name misspellings, wrong years, etc.) - based on history specific articles and history books plus feedback from key participants, I defined the key stories that must be told and the key participants that must be introduced in the book. (But I have over a hundred side stories that have consumed my time and the book!) - setup a subversion repo to track my book writing in LaTeX. Used comments to integrate notes in from all the histories I found, timelines (94 events), and integrated interviews. - used old Usenet / newsgroups archives to supplement stories. (Google Groups has been useful with its date-based searches.) I also found various archives available for downloads. In addition, even some newsgroup postings and old email archives were included in some of the historical BSD releases and CSRG archives. - found copies of old magazine articles and found archives old magazines at libraries. In addition, some contacts scanned documents or shipped documents to me. I also transcribed or took notes from several interviews (done by others). - found many old master theses, university (multiple schools) technical reports, BBN technical reports, legal documents, old (signed) software licenses, etc. that had much history details. - purchased every book that I learned about that contained BSD history and took detailed notes. (For what it is worth, to give you a size example of the most expansive BSD history in print: Salus's excellent Quarter Century book has two chapters totaling around 18 pages; and McKusick's most-interesting Open Sources chapter is around 7,300 words. Some of my chapters for short periods of time and for only single releases are longer.) I have been working on it for over five years. I try not to cover the side stories too much; for example I cannot cover in great detail the history of Unix, history of BBN's related work, history of open source licensing, nor DEC, PDP-11 or VAX history. (I have noticed that other participants on this group have written some of this related work and at least one document is a source in my bibliography -- Thanks Dave W.!) The amount of stories is amazing, such as connection between co-inventor of Unix and Berkeley, Berkeley Pascal, Virtual Memory Extensions for Unix "vmunix", VMS vs. Unix performance, DARPA research and founding of CSRG, TCP/IP and Internet services, support for multiple hardware platforms from one source tree, birth of Berkeley licensing and replacing proprietary software, the continuation of 2BSD (PDP-11), the Internet Worm, 386BSD and patchkits, birth of NetBSD and FreeBSD, AT&T/CSRG/BSDI/Novell legal proceedings, OpenBSD fork, and much more. Plus lots of interesting side stories: adventure and trek; INGRES database; SPICE2; birth of DNS; Email services (beyond and before SMTP); commercial Unix licensing and then open source licensing; first multi-processor support; revision control systems, BBN and Unix and ARPANET work; founding of Sun (and other BSD-based Unixes); and the intersection of BSD, Lucasfilm, Pixar, CMU Mach, NeXT, BSD again, Rhapsody/Darwin/MacOSX, etc. Lots of the stories are outside of University of California at Berkeley. Four other potential book ideas from my work include: history of open source licensing (as a side project I contacted the developers of the code containing the first "open source" style licenses known), history of X (Andrew window system, Verex, V window system, W window system, and then MIT wouldn't use Stanford's code so birth of "X", etc.), technical history of SUN and Sun Microsystems, and history of MacOSX (starting over a decade earlier, see my "intersection" above). I have missed several self-imposed deadlines for publishing the book. I have learned that every direction I take seems to bring in more never-ending stories (for example my recent research and interviews introduced the history of Ultrix), so I have broke it down into a few key things I need to cover and then ignore the many side stories and TODOs not fully researched to save them for a "second edition" of the book. My current work is to complete some integrations of my interviews and finish tying in the stories; pay for proofreading and editing for writing style; finish bibtex entries; send chapters to my many waiting reviewers; integrate their feedback; complete indexing; pay for cover artwork; and pay for another round of proofreading. I will reset my goal: get the book to my printer by end of August. If anyone has advice or suggestions for research methods for similar technical computer or software history books, it may be interesting to continue a discussion about it. I'd like to encourage any of you with an interest in the history of open source licensing, Unix, Berkeley Pascal, or BSD to contact me. Thanks for reading! Jeremy C. Reed echo 'EhZ[h ^jjf0%%h[[Zc[Z_W$d[j%Xeeai%ZW[ced#]dk#f[d]k_d%' | \ tr '#-~' '\-.-{' (This is a different book.) p.s For fun ... miscellaneous notes and todos: t1:bsd-history$ wc * | grep " total$" 30781 140722 2035160 total latex files: t1:bsd-history$ wc svn-bsd-history/*tex | grep " total$" 33088 223360 1425282 total t1:bsd-history$ egrep 'CITE:|\\cite{' svn-bsd-history/*tex | wc -l 1457 t1:bsd-history$ grep '\\bibitem' generated/book.bbl | wc -l 211 most of my interviews: t1:bsd-history$ wc interviews/{done,todo}/*-interview* | grep " total$" 15948 131526 750691 total my collection of related Usenet postings, technical reports, SCCS histories, old BSDs, etc: t1:bsd-history$ du -sm archives 5306 archives t1:bsd-history$ find -x archives -type f | wc -l 539001