By James Wallace Harris, Wednesday, October 12, 2016
Classics of Science Fiction version 4 is finally finished. I could not have done it without my friend Mike. I had been dreading doing all the work required to update the eleven year-old version 3. In fact, months ago I had given up on the idea, and just moved the v. 3 lists off my hosting site to this blog. I just didn’t want to relearn PHP and MySQL. Mike has been studying new programming languages like Go and Swift, and I asked him if he thought there was an easier way of doing the project. I told him I didn’t want to mess with a hosting site, a SQL database, or creating a data entry system. We finally decided on using .csv files for data entry, and typed everything in Notepad++. Mike programmed the project in Go. He then developed a complete template based system that generated all the .html files, which we copied to WordPress. It took us weeks to enter in the 65 lists used to create the new Classics of Science Fiction list. After all this studying of science fiction books we’re both looking forward to getting back to reading science fiction.
Titles and authors are linked to content on Wikipedia and Goodreads. Percentage ranks are linked to the actual lists each book were on. Unfortunately, this makes the main page rather large and slow to load.
I’m not sure how many science fiction readers care about such lists, but if you study the new site, it’s very revealing. I love studying how books become popular, and then how they are forgotten. The new main table which ranks the books by the percentage of lists each title is on, replaces the old method of just counting total lists. Not all books could have been on all lists, so to be fair, we calculated the percentage of lists they were on of all the lists they could have been on.
I know all this math might put some people off, so we show both percentages and counts on the tables were titles are ranked.
My favorite table is Versions 1-4 compared. Titles in red are those books that didn’t make it to version 4. Titles in blue are those new to version 4. This quickly lets you see what books are being forgotten, and which books are spreading in our memories.
Each time we make a new version of the Classics of Science Fiction list we love to see which newer books stand out in the crowd. This time we were anxious to see how many books by women writers had become popular in the last decade. We also made a separate list showing which books by women writers are the most remembered in all the lists we gathered. This time, we found eight lists devoted completely to women writers, but most of those lists didn’t try to rank titles. The Left Hand of Darkness by Ursula K. Le Guin was on 43 of the 65 total lists. But we figured it could have only been on 48 of those lists at most. This meant that The Left Hand of Darkness was on 90% of the lists it could have been on. 43 of 48.
Using percentages allowed newer books to rank higher than they did in version 3. We figure this is fair because we’re removing the penalty for being newer. Older books have an advantage under the system of just counting total lists. For example, Ancillary Justice by Ann Leckie was on just 11 lists, but because the maximum number of lists it could have been on was just 22 lists, it’s overall ranking was at the 50% rate.
Ultimately, meta-lists like these don’t mean much. It’s just a statistical way to track how books are remembered. But I find that fascinating. I’m sure fans will be outraged if their favorite science fiction novel doesn’t show up on the list. We didn’t choose these titles. Study the 65 lists we used. I’m sure you’ll probably find your favorite novel on some of them. For the main list, it took being on at least 10 lists. To create the list of books by women writers, we wanted it to be around 100 titles. That required using a cutoff of being on at least 4 lists.
We’ve learned that longer lists get boring for readers. I think ideal book list runs to around 100 titles, but for the main list we allowed it to run to 140. Version 3 ran to 193 titles and it was way too long. Probably for many bookworms, Top 25 or Top 50 lists are more appealing. Just scan down the ranked lists and stop when you feel titles are less than classic. The point of this project is to show which books other readers are remembering, and not to make any claims that these are the best books.
JWH
Auxiliary Memory 
v.4? Wow. I’ve only been around here for the last two months or so. This monument comes as a total surprise. I guess I’d better go back, rig the sails and drop a shiteful of nets out to catch anything related to JWH.
I can only hope that once I’ve caught up, I won’t be so intimidated as to decline to comment further.
Yeah sure.
jim
Lots of books from the 50s and 60s.
Great Job.
Some lists are unknown to me, others are pending add to Premios y Listas
<>
I used similar criteria: http://www.premiosylistas.com/2_listas/lst/PYL2015SFNN.html
I’m in awe of all the work you did! Just like the Stock Market, the popularity of books and writers rise and fall. Of course, the availability of the older SF books might have a lot to do with it. What will appeal to a young, new audience is always fun speculation.
That’s an interesting analogy – the Stock Market. I’m sort of shocked when I look at some lists of current best selling science fiction. Most of the titles and authors are completely unknown to me. Yet some of them will be the classics young people will remember when they get old. When I was young I used to love reading books by Sam Moskowitz. But I always thought he was obsessed with really old science fiction. Now I’m his age back then, and I’m hung up on really old science fiction. I’d love to be able to do some demographics on the age of books and the age of readers.
Will this list be made a list on WWE as well? That would be great…
That’s up to Dave. I don’t mind people using these lists elsewhere as long as they give us a link back to the original site.
I was surprised how low Gene Wolfe’s The Book of the New Sun was, but then I noticed that the first part of the series, The Shadow of the Torturer, was counted separately. Don’t you think those should be consolidated into a single entry?
Quizai, we’ve been agonizing over that for weeks. Voting for series has caused us problems. We wished that list makers wouldn’t allow votes for series, and ask for specific books. It’s allowing them for vote for multiple books with one vote. Can we assume the voters for Shadow of the Torturer love the other three volumes just as much? If so, then I suppose we could give the 10 lists for Shadow of the Torturer to The Book of the New Sun, and it would have 21. We could also break out all four books and give them 11 votes each, and then add 10 to Shadow of the Torturer, so it would have 21.
Which do you think is fairer? We’ll decide for version 4.1, when we regenerate the lists.
It’s tricky. I don’t think I could come up with a universal rule for this situation. Liking Dune, for example, is not the same as liking the whole series.
However in the case of Book of the New Sun, if you enjoy Torturer I think its very likely that you have read and enjoyed the entire series. It’s really one long cohesive book. I have to believe a vote for Torturer is a vote for BotNS, despite that not being the case with other series.
Mike and I have discussed this and have decided to add any votes for Torturer to Book of the New Sun. We’ll update the programming code soon and regenerate the lists. That means The Book of the New Sun will have 21 lists, and move up on our list.
Thanks for doing this. Truely awesome. I am happy to see Snow Crash finally made it. Do you think movies have an influence? Example, I Am Legend.
Mike and I have argued over the impact of movies on the public’s memory of books. I think a movie is essential in the long run. Mike thinks a great book can be remembered without the help of a film. I think many of the 19th century classics we remember today is because of all the movie versions.
Great Job.
Some lists are unknown to me, others are pending add to Premios y Listas
<>
I used similar criteria: <a href="http://www. Mike and I have discussed this and have decided to add any votes for Torturer to Book of the New Sun. We’ll update the programming code soon and regenerate the lists.