AI doomsday and AI heaven: live forever with AI God (LessWrong! Yudkowsky! Rationality!) www.youtube.com/watch?v=tAJI... - video pivottoai.libsyn.com/20250817-ai-... - podcast 19 minutes, sit yourself down for this one text version coming in a bit
AI doomsday and AI heaven: live forever with AI God (LessWrong! Yudkowsky! Rationality!) www.youtube.com/watch?v=tAJI... - video pivottoai.libsyn.com/20250817-ai-... - podcast 19 minutes, sit yourself down for this one text version coming in a bit
βToday Iβm going to make your life worseβ ππππ―π―π―
I am nearly certain I linked you fluxus.io/article/alch... this before: AGI [...] requires mathematics that doesn't exist to model biology we don't understand to implement functions nobody can define. That's "impossible" in any practical sense.
but that's the beauty of it: it *doesn't* matter. once you drive (unquestioned) the message that "AI is magic and can do anything" you don't need to justify anything. "climate change? AGI will solve it! how? doesn't matter!" (...) "AGI? evolved LLMs will figure that out! how? IT DOESN'T MATTER"
"evolved LLMs will figure that out" -- that's where Doctorow said no matter how fine prize horses you breed they will never give birth to a locomotive.
but what if it's a SDGH (Super-Duper General Horse)? uh?! better get on the good side of the SDGH *now*, or he'll kick you in the head for eternity
look bro next model totally will, are you some sorta ai *vegan*
I am. I lived through the 1987β2000 AI winter and vividly remember the claims. Funnily enough, one of the most important programs in the history of science came out in that period and if you squint the wrong way then it's AI: Tierra. And almost no one even knows what it was!
I donβt think I can ingest any more Chud-kowsky.
AI hells is the background to this book. I'm assuming that they didn't read it as the take is somewhat different. (The also failed lessons from Philip K Dick so YMMV on that) en.wikipedia.org/wiki/Surface...
Roko's basilisk was July 2010 and didn't popularise until years later, Surface Detail was October 2010, so it's quite likely neither was aware of the other
To be fair it's not a huge step from "super powerful AI" to "singularity" and all associated issues. Banks touched on it in most of his scifi work. Plus Dick, etc, historically. Have to say that it's utterly unsurprising that "Rationalists" took the dumbest take possible and went all-in.
FWIW as an Iain M Banks nerd the Silicon Valley twats fundamentally misunderstanding his books gives me real psychic damage.
Having read two of his books and then given up, I'd be interested in your take on what they are meant to say. My read was: super-AI is actually great, it will be benevolent, we should have it run government, and if you disagree, you are evil. That and his incoherence regarding AI are why I gave up.
The AI Minds rule by a sort of consensus. Pretty much *everything* is a game to them. Post-scarcity so no real jobs but organics find stuff that they do that they enjoy, and do that. If they get caught up in an AI's game then suck to be them. Intelligent life is, well, a bunch of pets.
Phlebas was his first SF novel where a lot of things were not fleshed out, and the Culture was seen from the outside. And pretty much every civ in The Algebraist sucks for one reason or another. The AI thing isn't really the point, just main chars civ being bigots.
Anyway, the fact that, generally, Culture citizens live lives of plenty doesn't stop them from being pets. In Phlebas, Horza realises this and wants no part of it, even though his side will lose. The Idirans aren't dumb, they just aren't AIs that will warcrime *their pets* until they win.
Thanks! So, Phlebas isn't representative, and his later novels become less "let's build the Torment Nexus, it will be amazing". Still, I think your read of Phlebas itself is too charitable. Fal 'Ngeestra is written as the most ethical, sensible character, and Horza and the Idirans as tragic fools.
A possibility is, of course, that Banks himself didn't consciously realise what he was signalling (including to singularitarians who would read it years later) with the way he wrote Consider Phlebas. Maybe he just got better at communicating and at expressing his thoughts as he gained experience.
Surface Detail contains literal AI hells that people go to when they die via mapping brian on death. Plus politics around whether this is moral, even if going there is part of your religion... I know that it's a bit "yeah but read eveything that x wrote" but you could have asked for specifics!
as I said, read Wasp Factory, it'll become clearer
I really don't want to annoy you, but I think @hairychris.bsky.social's argument of later Culture novels being salient makes more sense to me. I mean, if I say Rowling is a transphobe, a reading of the very first book she published wouldn't be a counter-argument, only her recent statements could be.
Also, he reads creepily voyeuristic whenever he writes about violence, war, and the destruction of large civilian infrastructure. Not: war is terrible, but: look how awe-inspiring this level of destruction is!
i would say read his literary fiction, start with Wasp Factory
I can try that sometimes, but it wouldn't change the world building of Consider Phlebas: the Idirans who don't want to outsource their decisions to AI are coded as the villains and depicted as stupid, although the largest war crime directly witnessed in the story is committed by the Culture.
The other one I read is The Algebraist. The civilisation suppressing AI is again depicted as the villains, but again the largest war crimes directly witnessed in the story are committed *against them*. Their own alleged evil is largely told, not shown. There's a weird pattern here, even at only n=2.
come back when you've read Wasp Factory
Fundamentally and wilfully.
cheers to @bvlsingler.bsky.social for productive pushback, I mention it in the episode