Reading Period: August 11 - September 25
1. How the Word is Passed (A), by Clint Smith
Link: https://www.goodreads.com/book/show/55643287-how-the-word-is-passed
Very emotionally resonant book, I could hear Clint's passion in every sentence. I liked Between The World and Me a lot better, a book with similar themes, but this one still packed quite the punch. This book is certainly very good. But it is first worth mentioning that it sustains an extremely high Goodreads rating, and with any book of this topic it's hard to know if this means anything. My worry with picking up a book like this is that I get hit with a book of little intellectual value that maintains high ratings due to audience capture or political signposting. The books of Robin DiAngelo and Ibram X. Kendi, for example, have been said to maintain their status not by their inherit merit but by their signaling. As another example, you can imagine not many moderates/leftists reads far-right literature, so reviews of far-right books will be heavily skewed/praiseworthy.
There were a few moments when the book clearly leans into modern political rhetoric, such as this statement: "Which meant he needed significant republican support. These were people who had built careers on political ideas that were harmful to black people." Both "build careers on" and "ideas" here, in particular, strike me as interesting. It is not surprising that Clint thinks modern republicans are bad (pro-policing, anti-DEI, economic status-quo), but Clint claims the "ideas" are harmful. People like Thomas Sowell would disagree, and there is enough nuance here that if not uncharitable, the claim is at the very least rhetorically ineffective. Clint also states that "I asked him if there would ever be an America in which White Americans were not actively working to keep themselves positioned atop the nations racial hierarchy." I am not certain what "actively working" means here, and it seems to convey that Clint sees modern racial issues as explicit oppression (rather than simply passive "working", like being against DEI). This is a totally fine statement for him to make; I include it because once I read it, I understood the book a lot better. The history of slavery has shaped Clint's view of the world, and the book is in a sense grappling with stories that explain to the reader how this legacy survives in the modern era (which is not far removed). Clint also states something along the lines of "did people come together in 1776 and make some pretty radical ideas, yeah", but actually this form of government is not novel and we cannot forgive the sins of the slave-owning founding fathers etc. I think he underrates the concept of individual liberty here, and how the US sparked a surge of democracy and revolution for the oppressed around the world, even if we concede the words here were written by imperfect and perhaps entirely immoral people.
Also, it's distressing to hear the rhetoric of groups like the "Sons of the confederacy" in this book, but one can argue that these people are bottom-of-the-barrel stupid and don't represent any significant slice of the American populace. A better tell for widespread racism is the fact that some crazy high number of white Americans (maybe around ~10%) wouldn't want their child dating someone of a different race. Now some of this may not be as racist as it seems (a Polish grandmother for example wanting to continue the Polish bloodline and not approving a marriage to an Irish-Catholic), but there's certainly a hell of a lot of racism. Clint sees the removal of statues of historical figures with Confederacy/slaveholding ties as important, and is distressed that there is resistance to this. I'd argue most people don't care and many just see it hard to draw the line (and removing George Washington and Thomas Jefferson statues would seem strange). I'm personally fine with removing Confederate statues, but I don't see it as a very important milestone in the greater fight against injustice, there are many other battles on this front I'd rather be fought. And this is coming from someone who is only lightly opposed to Sherman's March Part II, where us northerners take another victory lap and go through and burn the South once again.
The central error that I believe Clint makes is over-extrapolating historical injustice to political issues he currently cares about. In a society where virtually everybody dramatically and horrifically under-extrapolates this injustice, this is far from a sin. But I actually think it hurts his message quite considerably. In some ways we devalue the horrors of slavery when we equate them to modern criminal justice reform. It's not uncommon to compare the plight of slavery with those on death row, but it should be. I have lawyer friends who have spent countless hours pro-bono in Texas seeking to get innocent people off of death row, and its obvious that a lot of public uproar from these cases tends to be regarding those wrongly convinced. But Clint ignores this aspect, and draws many parallels between guilty criminals and slaves. Clint quotes a prisoner who states he believes he's "going through the very same thing folks fought and died for." Clint also details the gruesome death-penalty-deaths of a variety of individuals, and points out how quickly the jury deliberated on some of these cases as a point to continued current racism. But it's hard to avoid confronting the conservative talking point here. Which is this: guilty people on death row are not slaves, they are there because they murdered innocent women and children; drawing this parallel is unhelpful. I am not making this claim, but Clint makes no effort to counteract this obvious argument. Many of the death row case examples he gives don't survive this "Fox News" style layup: I looked up the most egregious case Clint highlights, and it was for a man who murdered his kids. Now, the current US prison system will certainly be looked upon with horror in future generations. In addition to racial injustice, the mere conditions in which an average prisoner has to endure (even if guilty) are horrific and distressing. In some sense this book is right up my political-viewpoint alley, but I think Clint stretches too much here in order to make a point.
This all being said, the book shines when discussing with unflinching honestly how horrific the institutionalized system of slavery was. And this discussion is so well written and well argued that I'd argue that this book should be required reading in every high school. Clint does a phenomenal job with his historical analysis that its hard not to feel a whiplash of emotions with every page. He states that slavery wasn't simply a dehumanization, it depended on human sentience. It required a subject that could be terrorized, and essentially oppressed and tortured into submission. This wouldn't work with a machine, the slaveholders needed the power dynamic of crushing a human soul in order to maintain such a horrific hellscape of immorality decade after decade. The sheer evil apparent in this system is so baffling, so unbelievably inexcusable, it makes me want to renounce my "humanity," whatever that is.
2. Letter from the Birmingham Jail (P), by Martin Luther King Jr.
Link: https://www.goodreads.com/book/show/203899.Letter_from_the_Birmingham_Jail
"We will reach the goal of freedom in Birmingham and all over the nation, because the goal of America is Freedom."
MLK is certainly an incredible writer. Well reasoned, convincing, and inspiring. He states that "Injustice anywhere is a threat to justice anywhere" and "We know through painful experience that freedom is never voluntarily given by the oppressor; it must be demanded by the oppressed." Similar to Civil Disobedience, MLK states that "I submit that an individual who breaks a law that conscience tells him is unjust, and willingly accepts the penalty by staying in jail to arouse the conscience of the community over its injustice, is in reality expressing the very highest respect for the law." He also states that the police officers cracking down on "illegal" protests are using "moral means to preserve immoral ends." What is more immoral, to peacefully protest segregation in the streets, or to arrest and break up these protests in order to uphold a wildly evil and immoral system of institutionalized racism?
I found MLK's religious rhetoric convincing, and resonated with a fairly long criticism he leveled against the church:
"There was a time when the church was very powerful. It was during that period when the early Christians rejoiced when they were deemed worthy to suffer for what they believed. In those days the church was not merely a thermometer that recorded the ideas and principles of popular opinion; it was a thermostat that transformed the mores of society. Whenever the early Christians entered a town the power structure got disturbed and immediately sought to convict them for being 'disturbers of the peace' and 'outside agitators.' But they went on with the conviction that they were a 'colony of heaven' and had to obey God rather than man. They were small in number but big in commitment. They were too God-intoxicated to be 'astronomically intimidated.' They brought and end to such ancient evils as infanticide and gladiatorial contests. Things are different now. The contemporary church is often a weak, ineffectual voice with an uncertain sound. It is so often the arch supporter of the status quo. Far from being disturbed by the presence of the church, the power structure of the average community is consoled by the church's silent and often vocal sanction of things as they are."
3. Sixth of the Dusk (P), by Brandon Sanderson
Link: https://www.goodreads.com/book/show/23397921-sixth-of-the-dusk
Quite the change of pace. Another great novella from Sanderson, not much else to say.
4. Pale Blue Dot (P), by Carl Sagan
Link: https://www.goodreads.com/book/show/61663.Pale_Blue_Dot
It took me a while to finish, but overall I really enjoyed the book. The first chapter is one of the best chapters of non-fiction I've ever read. Carl makes the claim that: "On the scale of worlds - to say nothing of stars and galaxies - humans are inconsequential, a thin film of life on an obscure and solitary lump of rock and metal." He also states that: "If I had to guess - especially considering our long sequence of failed chauvinisms - I would guess that the Universe is filled with beings far more intelligent, far more advanced than we are." I also really liked his insistence on space exploration, and what it means for humanity: "We lack consensus about our place in the Universe. There is no generally agreed upon long-term vision of the goal of our species - other than, perhaps, simple survival." It's hard to not want to reach for the stars after reading: "Their eventual choice, as ours, is spaceflight or extinction."
However, the book gets lost a bit in hard science. This is fine, but the material is fairly old so there is just better content out there at this point (A Short History of Nearly Everything comes to mind). Still, I found Carl's discussion of nuclear war fascinating. It's always interesting to read the perspective of an expert in one field who is writing a novel comment on the current events of the time. Carl states: "Not only do we often ignore the warnings of the oracles; characteristically we do not even consult them." And: "Nuclear weapons were invented in 1945. It took until 1983 before the global consequences of thermonuclear war were understood." I will close with one of the most surprising parts of the book, when Carl analyzes a phenomena I've always felt but couldn't quite put my finger on. Unfortunately, it looks like nothing much has changed with the passage of time:
"We are sometimes told that this or that invention would of course not be misused. No sane person would be so reckless. This is the 'only a madman' argument. Whenever I hear it (and it's often trotted out in such debates), I remind myself that madmen really do exist. Sometimes they achieve the highest levels of political power in modern industrial nations. This is the century of Hitler and Stalin, tyrants who posed the gravest dangers not just to the rest of the human family, but to their own people as well. In the winter and spring of 1945, Hitler ordered Germany to be destroyed - even 'what the people need for elementary survival' - because the surviving Germans had 'betrayed' him, and at any rate were 'inferior' to those who had already died. If Hitler had nuclear weapons, the threat of a counterstrike by Allied nuclear weapons, had there been any, is unlikely to have dissuaded him. It might have encouraged him."
5. Deep Utopia (P), by Nick Bostrom
Link: https://www.goodreads.com/book/show/208707460-deep-utopia
The most important philosopher of our time, who wrote a terrifying book about the implications of advanced AI, pens a book about what we will do with all our time in utopia when AI solves every problem ever. It's a baffling change of pace, but quite needed and refreshing. To summarize, Bostrom asks "at technological maturity, is there anything that cannot be done better by machine?" The answer, is basically no. Nick says "For it is quite conceivable that, at technological maturity, you would in fact be slightly harming your child whenever you indulged in some DIY parenting." Even moral status and cognition, at some point, may not be exclusively biological traits. What then? He defines a digital mind as: "A mind implemented on a computer. Could for example be an upload of a human or animal mind, or an AI of a design and sophistication that makes it a moral patient, i.e. one whose welfare or interests matter for their own sake." I've spent more time thinking about digital minds than almost anyone alive, and yet I found this insight entirely novel:
"If we imagine - as I tend to do - a future that is mostly populated by digital minds, then the convertibility of wealth into well-being becomes even clearer. Digital minds, be they AIs or uploads, need computation. More computation means longer life, faster thinking, and potentially deeper and more expansive conscious experiences. More computation also means more copies, digital children, and offshoots of all kinds, should such be desired."
I had no idea Nick was funny. In fact, he's hilarious. The book mentions DJs dropping "bangers" at nightclubs, has literal illustrations of memes, and quotes like: "Millions long for immortality who don't know what to do with themselves on a rainy Sunday afternoon." My favorite quote was this: "This is especially clear if we entertain the radical possibility that we are not in a simulation." Nick also takes aim at sports fanatics, and golf: "Well, it is actually unclear why people play golf. The real reason might be to have fun - and if so, then, of course, utopia would offer the more efficient option of wireheading as a means of obtaining pleasure." He illustrates the weirdness of human wiring further here:
"More people jump out of their seats when their soccer team scores a goal than when an international agency publishes a report saying that a hundred thousand fewer children died form preventable diseases this year than last. (We take this to be completely normal, but I wonder, if we could see ourselves through the eyes of angels, whether we would not recognize in this pattern of excitement and indifference something quite perverse - the warped sentiments of a moral degenerate? Is it not, implicitly, a sort of emotional middle finger to the suffering and desperation of other sentient beings?)"
This book is genuinely novel. It explores not post-scarcity worlds (which have been done in fiction), but rather post-instrumental worlds. "Post-instrumental utopia: No instrumental need for any human effort. Implies post-work but goes beyond in also assuming no instrumental need for any non-economic work either - no need to exercise to keep fit, for example, no need to study to learn,; no need to actively evaluate and select in order to obtain the kinds of food, shelter, music, and clothing that you prefer. This is a far more radical conception than the preceding three types of utopia, and has been much less explored."
Nick also discusses the meaning of life at length, although he mostly admits ignorance: "Journalists often ask me if I can share some life advice for young people. I'm flattered to be asked, but this really seems to me like asking somebody who is still playing his first-ever game of chess to give advice on chess strategy." I found it interesting to contemplate Nick's linguistic choices and deep exploration of words like "desire." When contemplating meaning, should we care about what we desire, or what we should desire? Nick claims that there is a spectrum from "What I actually right now consciously desire" to "What I would desire if I had the character that I wish I had" to "What I would desire if my desires perfectly tracked objective truths about what is impersonally 'best for the world'." Which should we care about? Should we only care about realized utility? Or should what actually happens (even if not consciously realized by anyone) matter? Example: "Well, for example, I would not want my wife to have an affair with her tennis instructor. Even if I would never find out about it. I don't want my life to be based on a big illusion."
It's hard to review this book. There are so many ideas and insights flying around that it's hard to keep track of them all. Nick's narrative choices, reminiscent of Godel, Escher, Bach are interesting, but largely ineffective. The ending falls flat on a confusing narrative note. There is a extremely long story about a room heater turned conscious, which largely misses the mark except for the closing statement: "Is ThermoRex's life better or worse than that of the median human being who has lived on Earth so far?" But the book is full of life. Nick randomly bursts out in existential terror randomly during his narration (which I loved), and he doesn't stop his philosophical musings on the human condition: "I wondered why I had been made with a soul that had the capacity to wonder but not the capacity to find out; why I could see so much that was wrong while seemingly being unable to do anything about it." If I had to guess, I'd guess that Nick wrote this book more as an excuse to think through a ton of interesting ideas. His plan wasn't: think really hard, form concrete world-changing ideas, and then meticulously think through how to convey this to the public. His plan was: think really hard, and write a book as a companion to this process in order to have a planned start and finish line.
Overall, I thoroughly enjoyed it. This book will fall completely flat for most people for a variety of reasons, but to me it felt like reading the inner monologue of someone way smarter than me, who is thinking through bigger ideas than I've ever had the guts to tackle. Also, I am persuaded by almost everything Nick says, and I'm not quite sure what this means. But it's like I have been exposed to a radical, likely true, conceptualization of the world that is missed by almost everyone else. I'll end with this quote, and idea that I've been dying to verbalize in such a concise way. An idea that I hope is broader conveyed as we approach the sort of futures Nick discusses.
"I am inclined to say tough luck to the tragedy-lover. Or rather: feel free to get your fix from fantasy, or from history - only, please, do not insist on cooking your gruesome entertainment in a cauldron of interminable calamity and never-ending bad news. It is true that good books and films have been inspired by wars and atrocities. It would have been better if these wars and atrocities had not occurred and we had not had these books and films. The same applies at the personal scale. People coping with the loss of a child, dementia, abject poverty, cancer, depression, severe abuse: I submit it would be worth giving up a lot of good stories to get rid of those harms. If that makes our lives less meaningful, so be it."
6. Deep (P), by James Nestor
Link: https://www.goodreads.com/book/show/18222705-deep
A book about a bunch of topics, namely freediving, the deep ocean, and marine life. Freediving is a crazy sport. It's a simple concept, you attach yourself to a weighted sled, which shoots you straight down hundreds of feet into the a body of water. You black out. Then, an air balloon fills up and propels you back up to the top of the water, where you are resuscitated. You likely die. If you go far down and back up, you consider this a great accomplishment.
It's hard to read these freediving sections, because everyone is just blacking out and dying all the time. The parts of the book about the deep ocean are interesting, but a bit tiresome. And the parts about marine life are a little bit cooky. James says things like "Human language is analogue; sperm whale language may be digital." Most of the book is forgettable, but one unforgettable story is that of a researcher named Lilly, who encouraged a research assistant to become sexually explicit with a dolphin. This "research" lasted a while and got pretty weird, and needless to say "Lilly basically ruined the field for the next thirty years." My biggest problem with the book is that much of the narration is a bit unreliable/unbelievable, and certainly biased against "crude" humans and towards "advanced" aquatic life, which is what you'd expect from someone who writes a book about the ocean. Not quite my type of book, but maybe I don't care enough about the ocean.
7. The Three-Body Problem (A), by Cixin Liu
Link: https://www.goodreads.com/book/show/20518872-the-three-body-problem
You know, I really thought I'd love this one. It was fine, decent even, but nothing particularly special. Some of the sci-fi aspects are a bit too weird/ambitious, and I don't think the characters or writing quality stands out enough from other books. The concept largely carries this one to being worth the read.
8. If Anyone Builds It, Everyone Dies (P), by Eleizer Yudkowsky and Nate Soares
Link: https://www.goodreads.com/book/show/228646231-if-anyone-builds-it-everyone-dies
I've been waiting in anticipation of this book for quite a while. In my mind, and no one else's, I see IABIED as a direct competitor to my book earlier this year, Mind Crime, which deals with very similar subject matter. In fact, I spent quite a bit of advertising dollars linking my work to the IABIED pre-order Amazon page, money now forever gone into the ether with no certainty of impact. In some sense, I now feel that the torch has been passed, and I am unlikely to spend any more on ads (at least until the MIRI-led marketing machine fizzles out, or until we see more concrete signs of AGI's arrival).
The clearest distinction between books is that this book is way more accessible than mine. Eleizer has spent 20+ years thinking about this topic and refining his arguments, although I would argue his worldview hasn't changed much. His argument is fantastically simple: ASI is going to kill everyone. There is no complex theory to be discussed; some future AI model with "go hard" and this will result in the death of everyone one Earth. For the layperson, this is clearly the better book to read. It's also likely the better option for a trailblazing, likely net positive emergence of AI-related concern in the public sphere. This book is truly an accomplishment, and I couldn't be happier that it has a chance to run its course. Despite its flaws, I'm excited by that prospect that this will get more people thinking about ASI alignment, the greatest issue to have ever faced the human race.
Given that I've written an extremely niche, well researched, and complex book about the moral implications of artificial superintelligence, I am not the target audience here. The clearest example via this quote: "In other words, o1 went hard. It behaved as if it wanted to succeed." Whereas Mind Crime skipped over the info-hazard flags and went straight to discussing how ASIs could blackmail their way into power, Eleizer sticks with: "Paying people is a classic way of convincing them to do something." I have a chapter dedicated to the interconnected complexity of potential AI sentience and ASI takeover, and Eleizer mentions that an unaligned ASI named "Sable" may find it useful to promote "the idea of AI sentience and AI rights." Interestingly, I wonder if Sable will quote me? There isn't much insight into how the world might change here, Eleizer simply states: "If a horse starts costing more to feed than it can produce in labor, the horse is sent off to the glue factory." There are understandable trade-offs being made here in terms of messaging, and honestly I'm glad that the book is so different and "dumbed-down." It means that both I should no longer care about a broad audience, and also I didn't waste two years of my life writing a book that was replaced by a more popular, exact replica.
I disagree pretty strongly with a lot of the book. In some sense, many of my ideas are a direct response to thinking through many of the points raised here by the MIRI team. For example, I believe overindexing on extinction is wrong, and I'm not a fan of how this is handled in this book. It used to be overnight nanobots, but now dyson swarms, which blot out the sun, are an option for humanity's eventual demise. How long does this extend the timeline? How long does it take to build a dyson sphere? If it takes hundreds of years for humanity to die out because all sunlight is blocked by AI-built satellites, is that really what "extinction" means? I guess so, but some of this rhetoric seems like desperate stretching of "foom" => "doom" scenarios.
Also, Eleizer and Nate state that: "We consider interpretability researchers to be heroes." Thanks! But also that "learning to read some of an AI's mind is not a plan for aligning it, any more than learning what's going on inside atoms is a plan for making a nuclear reactor that doesn't melt down." This feels like a complete false equivalence. Sure, it may be true that interpretability research never gets good enough to be a reliable lie detector. But if it does, would this not make the alignment problem immensely easier?
There are some really interesting points in this book. This specifically: "Hinton has said that he actually thinks that it's more than 50 percent likely that AI will kill us, but he usually avoids saying this 'because there's other people who think it's less.'" Also, the ending quote resonated with me a hell of a lot: "May we be wrong, and shamed for how incredibly wrong we were, and fade into irrelevance and be forgotten except as an example of how not to think, and may humanity live happily ever after." Additionally, I think their whole framing of just-literally-say-what-you-believe is super helpful and inspiring. A lot of AI safety people seem to be 4D-chess-ing themselves into irrelevance. Here is Eleizer's comment: "We've watched this sort of thing play out for a while, with people not stating the real reasons for their proposals or why they think they have to be passed so urgently. And we've watched perfectly reasonable lawmakers smell something rotten and throw the whole package out." Why does no one think about ASI extinction risk? Maybe because everyone who does is too afraid to say it out loud, and thus no one says anything. They're correct here, it's time to change.
The key policy proposal in this book will be widely disregarded/hated:"So the safest bet would be to set the threshold low - say, at the level of eight of the most advanced GPUs from 2024 - and say that it is illegal to have nine GPUs that powerful in your garage, unmonitored by the international authority." Basically, we need to ban all AI research, lock up the GPUs, and form a world government that ensure no ASI is ever built. People will say it's unrealistic, but honestly, if you agree with Eleizer's priors there's just simply no other option. I'm not quite there yet, but there's a non-negligible chance that he's just correct about this problem and this is clearly the only solution.
Now, I'm going to end on sort of a strange criticism. Eleizer and Nate, in the final page, drop in the C.S. Lewis quote: "If we are all going to be destroyed by an atomic bomb, let that bomb when it comes find us doing sensible and human things - praying, working, teaching, reading, listening to music, bathing the children, playing tennis, chatting to our friends over a pint and a game of darts - not huddled together like frightened sheep and thinking about bombs." Their message ends on, what I have to assume, is some sort of attempt at optimism, where you're supposed to read this book and then go back to living life before it all ends for everyone? They state: "And once you have done all you can do? Live life well." I understand that most people won't have an impact on the outcome of ASI development. But this is sort of a "walk-it-back" kind of way to end such an important book. I get why they felt they had to do it, but it felt weak-willed and cowardly in an otherwise assertive book. Sure maybe don't get too crazy with it, but it seems obvious that the logical conclusion to agreeing with this book should be less "play tennis" and more "fight until the very end." Just a style preference, but I'd rather be encouraged to fight.
9. Deep Learning (P), by Ian Goodfellow
Link: https://www.goodreads.com/book/show/24072897-deep-learning
Clearly the most comprehensive, most helpful, and most rewarding ML book I've ever read. I read the first 200 pages a few years ago, and then recently in the span of a few weeks relentlessly pushed onwards to the 800 page finish line. It's funny finishing a technical book like this, as I feel I finally have a competent mental model of the subject matter. Without knowing the breath of the deep learning space, it's pretty impossible me to conceptualize the depth. Boltzmann machines (and energy models in general) still confuse me, but essentially all of the other DL lingo is now etched into my brain. That's what is great about finishing an 800 page book, there's no escaping a staggering familiarity with the topic at hand. This is probably the most important step I'll take in clearing the hurdle towards better technical understanding of AI, and I'm glad to have this behind me as the foundation.
10. The Dark Forest (A), by Cixin Liu
Link: https://www.goodreads.com/book/show/23168817-the-dark-forest
Again, a mix of good and bad. I like the storyline a lot, and it is true, hardcore sci-fi (which I'm a big fan of). Similar to the first book, there's a lot of early build up, with a semi-climax where a piece of incredibly cool and deadly technology takes the stage. It really does drive home the point that technological disparity amounts to genocide. However, a lot of the writing decisions are a bit bizarre. I don't think it's a cultural thing; the plot line/constant references to characters with "imaginary" lovers was pretty bizarre and concerning. I wasn't quite sure what I was reading at points, it was so weird and immersion breaking. Thankfully I chose not to put the book down, and it got back to sci-fi. Well, mostly. The annoying, repetitive patterns of humanity (changing public opinion at the drop of a hat, always predictably and in the most frustrating/unrealistic way possible) and the strange actions of the characters (no one ever says anything useful or competent) was frustrating as well. If 40% of the book (mostly that stuff) was cut, this would have been a really good book. Unfortunately it was just ok.
