'Machines of Loving Grace' - Key Highlights. 'All the 21st Century ... by 2036.'
Added 2024-10-13 19:05:59 +0000 UTCThe CEO of the one of the leaders in the race to human-level AI (Anthropic), envisions ‘Machines of Loving Grace’, watching over us. Dario Amodei weaves in hundreds of predictions in biology, neuroscience, economies and global politics, all premised on the ‘optimistic scenario’. Released less than 48 hours, I give you the full highlights, and my own analysis. Not for the faint-hearted.
Download Link: https://drive.google.com/file/d/1XxBiUlCqRjnwDPe8Nf_jcTTELkzuwh90/view?usp=sharing
Essay: https://darioamodei.com/machines-of-loving-grace
Poem: https://allpoetry.com/All-Watched-Over-By-Machines-Of-Loving-Grace
Competition vs Cooperation, The Goddess of Everything Else, Scott Alexander: https://slatestarcodex.com/2015/08/17/the-goddess-of-everything-else-2/
Anthropic Chief of Staff 2027: https://www.palladiummag.com/2024/05/17/my-last-five-years-of-work/
Socialist Calculation debate: https://en.wikipedia.org/wiki/Socialist_calculation_debate#:~:text=The%20socialist%20calculation%20debate%2C%20sometimes,of%20the%20means%20of%20production.
AI Drugs and Vaccines in Phase I: https://www.sciencedirect.com/science/article/pii/S135964462400134X
AlphaProteo: https://deepmind.google/discover/blog/alphaproteo-generates-novel-proteins-for-biology-and-health-research/
https://arxiv.org/pdf/2409.08022
Three-body problem: https://en.wikipedia.org/wiki/Three-body_problem
Wolfram, What is ChatGPT Doing: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/
BrainLM: https://www.biorxiv.org/content/10.1101/2023.09.12.557460v1.full.pdf
Adam D-Angelo Prediction: https://x.com/adamdangelo/status/1627726566259318784?lang=en
Prisoner’s Dilemma: https://en.wikipedia.org/wiki/Prisoner%27s_dilemma
China Nukes: https://thehill.com/opinion/international/4743139-china-ai-nuclear-weapons/
Comments
I find it a little eye-opening and also a little calming, that such a calm and level-headed person such as you also thinks about the possibility of bad outcomes, such as the (I believe half-jokingly) mentioned Chinese drone swarm.
pavelkomin
2025-02-15 09:59:03 +0000 UTC(duplicating my previous comment without the link in case the link hides it from being visible:) Now to test if I can leave links on Patreon comment threads: [no link included in this version of the comment] The above linked spreadsheet ("Notable Peoples' Views on AI - AI Quotes [Public]") is a collection of quotes I've started making with quotes on AI timelines, AI risk, and the impact of AI. The two quotes I referenced in my above comment are Quotes #23 and #36.
William
2024-12-16 06:11:34 +0000 UTCNow to test if I can leave links on Patreon comment threads: https://docs.google.com/spreadsheets/d/1u496oighD1qMnlfKIKYWeGEHwLMW-MugDocN4r1IHcE/edit?gid=0#gid=0 The above linked spreadsheet ("Notable Peoples' Views on AI - AI Quotes [Public]") is a collection of quotes I've started making with quotes on AI timelines, AI risk, and the impact of AI. The two quotes I referenced in my above comment are Quotes #23 and #36.
William
2024-12-16 06:10:51 +0000 UTCIn Philip's Dec 12th 2024 video "Never Browse Alone? Gemini 2 Live and ChatGPT Vision" he says that "Dario Amodei predicted transformative AI by 2026" at 10:00 in the video. I'm quite confident this is incorrect. In this essay, TMoLG, from Oct 1, Dario wrote "I think it [powerful AI] could come as early as 2026, though there are also ways it could take much longer." "As early as 2026" and "by 2026" mean very different things. "By 2026" is therefore a clear mischaracterization of Dario's view. Dario also gave an interview with Shirin Ghaffary of Bloomberg ("Bloomberg: Anthropic CEO Thinks AI May Outsmart Most Humans By 2026"), published two weeks after his essay, and comments on the 2026 number in it. Article excerpt: ---- In his essay, Amodei writes we might reach this milestone — which he prefers to call “powerful AI” rather than the often-used “AGI,” or artificial general intelligence — as soon as 2026. “I think that’s the earliest,” he clarified in our interview. “But you know, that doesn’t mean I think that’s necessarily the most likely… I can tell you a story where things get blocked and it doesn’t happen for 100 years. That is possible. But I would certainly bet in favor of this decade.” ---- The most ambiguous aspect of Dario's statement is whether "But I would certainly bet in favor of this decade" means that he assigns >50% chance to "this decade" (before 2030) or not. It's unclear to me, because it's possible to bet on things that are <50% likely as long as the market probability is even lower. E.g. Maybe Dario thinks that others say (to make up a number) 10% by 2030 and he thinks 30% by 2030, so he'd definitely want to bet on the AI will happen side of a bet against those people. While the above interpretation is possible, I think it's more likely that Dario actually means that it is >50% likely that we will get powerful AI before 2030. ("this decade" is also potentially ambiguous, though less so. I assume "this decade" means "by 2030," but conceivably he could have meant "within the next 10 years".) From these two Dario quotes, I infer that Dario's "powerful AI" timelines: - Have a long right tail with some probability stretched out over the next 100+ years - Have very little probability mass before 2026 (<1%?) - Probably has a median timeline of ~2028-2030. - Perhaps probably has a modal outcome slightly before the median outcome (perhaps 2027-2029), since that tends to be the case when probability density functions have this shape. - Probably has a mean expectation of much >2030 due to the long right tail, with some probability of not getting powerful AI for a few decades or a century increasing the mean by a lot. If my inference is significantly wrong due to to making false assumptions about the ambiguities in Dario's statements, it's very likely to be wrong in the direction of interpreting Dario's timelines as being too *short* rather than too long. For example, it wouldn't surprise me too much if Dario's modal outcome is before 2030, but his median is actually in the mid-2030s, and he just said "But I would certainly bet in favor of this decade" because the individual year or years with the most probability mass (his modal years) are in this decade (e.g. 2028-2029).
William
2024-12-16 06:08:13 +0000 UTCrelated to the topics (04:40:17) So another interesting hypothesis is the super superposition hypothesis. Can you describe what superposition is? and 3:54:43 – AI consciousness see this article I wrote related to superposition and Of Cats, Consciousness, and Cosmic Questions -The Great Escape of Schrodinger's Determined Human-Cat Duo – Bob and Spike | https://www.linkedin.com/pulse/cats-consciousness-cosmic-questions-the-great-escape-doug-hohulin-a6kuc/
Doug
2024-11-13 01:11:14 +0000 UTChttps://lexfridman.com/dario-amodei-transcript
Doug
2024-11-12 23:59:22 +0000 UTCTranscript for Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity | Lex Fridman Podcast #452 https://lexfridman.com/dario-amodei-transcript Here are the loose “chapters” in the conversation. Click link to jump approximately to that part in the transcript: 0:00 – Introduction 3:14 – Scaling laws 12:20 – Limits of LLM scaling 20:45 – Competition with OpenAI, Google, xAI, Meta 26:08 – Claude 29:44 – Opus 3.5 34:30 – Sonnet 3.5 37:50 – Claude 4.0 42:02 – Criticism of Claude 54:49 – AI Safety Levels 1:05:37 – ASL-3 and ASL-4 1:09:40 – Computer use 1:19:35 – Government regulation of AI 1:38:24 – Hiring a great team 1:47:14 – Post-training 1:52:39 – Constitutional AI 1:58:05 – Machines of Loving Grace 2:17:11 – AGI timeline 2:29:46 – Programming 2:36:46 – Meaning of life 2:42:53 – Amanda Askell 2:45:21 – Programming advice for non-technical people 2:49:09 – Talking to Claude 3:05:41 – Prompt engineering 3:14:15 – Post-training 3:18:54 – Constitutional AI 3:23:48 – System prompts 3:29:54 – Is Claude getting dumber? 3:41:56 – Character training 3:42:56 – Nature of truth 3:47:32 – Optimal rate of failure 3:54:43 – AI consciousness 4:09:14 – AGI 4:17:52 – Chris Olah 4:22:44 – Features, Circuits, Universality 4:40:17 – Superposition 4:51:16 – Monosemanticity 4:58:08 – Scaling Monosemanticity 5:06:56 – Macroscopic behavior of neural networks 5:11:50 – Beauty of neural networks
Doug
2024-11-12 23:57:04 +0000 UTCsee this poem Interwoven Destinies: An Exploration of Nature, Man, and AI https://www.linkedin.com/pulse/interwoven-destinies-exploration-nature-man-ai-doug-hohulin/ Tiger got to hunt, bird got to fly; Man got to sit and wonder 'why, why, why?' AI got to learn, adapt and grow; Seeking answers to questions we don't know. Tiger got to sleep, bird got to land; Man got to tell himself he understand. AI got to process, analyze, and thrive; Together with man, making the world alive. In this dance of life, we all play a part, Tiger, bird, man, and AI, with wisdom and heart. Together we'll conquer, with courage and grace, The challenges we meet, in this vast cosmic space. ― Kurt Vonnegut & Doug Hohulin & ChatGPT-4
Doug
2024-11-12 17:19:52 +0000 UTCI highly recommend the essay 'Meditations on Moloch' by Scott to understand the darker prospects for AI
Vlad Gheorghe
2024-10-27 18:22:47 +0000 UTCThat's true, but if I am going to stretch the point a bit, I think one thing people frequently forget is that although humans now learn novel things from much sparser amount of data, there is a vast evolutionary dataset that was needed to train the "base model" of our brain. The fact that humans learn differently from LLMs seems clear, but the fact that our brains have been pretrained over the course of hundreds of millions of years is a very real factor. There is a lot of implicit information stored in the anatomical conformation of our brain, the specifics of cell biological function and so on. This point is a little bit exotic, and perhaps pushing it a little in the context of this discussion. But I think it is an interesting perspective to keep in mind.
Sebastian
2024-10-25 12:21:52 +0000 UTCempathy i think even humans are bad at proving they have. Case being all HR and C-Suite executives whenever you talk to them. AI might not provably have same proprioceptive body maps like us which emotions can be projected on like humans do, but they can mimic emotionally intelligence much better than expert therapist i feel. My therapist even if she its just phonecalls, i think its enough to feel emotionally connected
Prashant Maurice
2024-10-20 14:38:38 +0000 UTCCool video, although I don't think he was saying that the 5-10 years would necessarily start from 2026, but from the "reasonably soon" that he mentioned afterwards. (He said 2026 or perhaps much later, and then offered "reasonably soon" as a compromise "for the purposes of this essay".)
Dark_Eternal
2024-10-18 22:36:35 +0000 UTCThat is a good point. But whatever model runs in our brain, it does things that LLMs haven't even touched. So far, all they are doing is reproducing stuff that is part of their training data, with the occasional lucky punch where they find some connection that humans (probably) haven't made yet. Which is just natural given the vast amount of data that we have produced.
Markus Heinsohn
2024-10-16 11:43:11 +0000 UTCThe only issue with this is that we do not really understand the underpinnings of our own cognition. Do humans do "true logical reasoning"? Are we not probabilitic? The atoms making us up certainly are, at least. Keep in mind that what you perceive of your own mind is completely unreliable. We think we see a coherent world, but in actuality our attention can only process a single visual object at a time, and the rest is just buffered placeholders that are updated as we move our attention there. This is the scientific consensus of the reality of perception, but no amount of introspection would give you the same answer. Who is to say that our perceived "reasoning" isn't just memorised patterns running on top of smoke and mirrors? The point is that we do not currently truly know the limits of a probabilistic system. For all we know we might be one ourselves.
Sebastian
2024-10-16 10:48:47 +0000 UTCThanks for sharing. I find even the best case scenario pretty scary as it is much easier to see how things could go wrong (AI arms race, loss of jobs, increased surveillance) and harder to see how the world could adapt to these brutal changes. This is what Claude had to say on the blindspots of its creator, pretty spot on I think: “Here are some possible blindspots in the essay: 1. Uneven distribution of benefits: The essay touches on global inequality, but may underestimate how technological advancements often exacerbate existing inequalities. There's a risk that the benefits of AI could be concentrated among already-privileged groups or nations. 2. Environmental impacts: While climate change is briefly mentioned, the essay doesn't deeply explore the potential environmental consequences of rapid AI-driven development and increased resource consumption. 3. Cultural and ethical diversity: The vision presented is largely based on Western liberal democratic values. It may not fully account for diverse cultural perspectives on progress, governance, and the role of technology in society. 4. Potential for misuse and weaponization: While risks are acknowledged, the essay might underestimate the potential for AI to be used maliciously by bad actors, both state and non-state. 5. Privacy and surveillance concerns: The essay doesn't deeply explore how increased AI capabilities might impact personal privacy or enable more pervasive surveillance. 6. Job displacement and economic disruption: While the essay touches on this, it may underestimate the short to medium-term economic disruption that could occur as AI rapidly changes the job market. 7. Psychological and social impacts: The rapid changes envisioned could have profound psychological impacts on individuals and societies. The essay doesn't deeply explore how humans might cope with such rapid transformation. 8. AI autonomy and control: The essay assumes that humans will remain in control of AI systems. It doesn't fully explore scenarios where AI systems become increasingly autonomous or difficult to control. 9. Geopolitical power shifts: While the essay discusses an "entente strategy," it may underestimate how AI could radically reshape global power dynamics in unpredictable ways. 10. Overreliance on AI: There's a potential blindspot in not considering the risks of becoming overly dependent on AI systems for critical societal functions. 11. Biological diversity and evolution: The focus on human enhancement doesn't address how widespread genetic modifications might impact biological diversity or human evolution. 12. Philosophical and existential questions: The essay doesn't deeply explore how such radical changes might impact our understanding of what it means to be human or our place in the universe.”
Eddine Maiza
2024-10-16 05:00:56 +0000 UTChttps://notebooklm.google.com/notebook/95e63a56-6cec-4708-9ce1-87e82f40c45f/audio?pli=1 from NotebookLM here is AI breaking down its thoughts on "Amodei's" mostly optimistic essay.
Lee FRASER
2024-10-15 23:05:40 +0000 UTCThe problem is, that no matter what you do, it's still based on a probabilistic system, and not true logical reasoning. Sure, if it gets things right often, it is useful as a tool (I use LLMs to help with some software development tasks). But I believe we're still a very long way from proper reasoning AI systems which are also creative.
Markus Heinsohn
2024-10-15 06:05:02 +0000 UTCThank you Philip - I have already watched your video twice and read the essay. And I have set aside time with my husband to watch it together and discuss what this possible/probable near future means for us and our 11 yo daughter. I must say I’m a bit in shock
Sean Gallagher
2024-10-14 20:24:59 +0000 UTCWow, what an inspiring video. Thank you!
generousB
2024-10-14 19:55:27 +0000 UTCGood video as always, though maybe it focused a bit too much on the 2026 number? To me, it felt like Dario wasn't so sure about the exact timing of "powerful AI" and 2026 was just the fastest he felt it could possibly happen, not that he thought it was likely that year. The essay seems much more in the vein of "whenever it does happen, this is what will happen in the years following". Having said that, the fact that he thinks 2026 is even a possibility is pretty telling in itself. I'll be very curious to see what releases in the next 6 months. If there are more "step change" level models, that'll be a strong signal that there is more to come.
Shawn Fumo
2024-10-14 19:19:45 +0000 UTCI'm not sure about that paper (the conclusions feel overstated). To me, this one is more interesting (which I think Philip mentioned in a video at one point): http://www.arxiv.org/abs/2409.13373 In that, they find that 01 dips a bit on "randomized mystery blocksworld" as compared to regular mystery blocksworld, but still manages to solve it 37% of the time (and must by definition not be in the training data). That's compared to other models that get 0%. If you scroll to the bottom of the paper to see what that actually entails, it certainly would take a bit for me to figure out. This is a block stacking style problem but with actions and objects that are just long random strings of numbers and letters.
Shawn Fumo
2024-10-14 19:04:32 +0000 UTCI still believe generative AI is stalling, and it will take more than GPT o1-level trickery to come up with true reasoning and innovating AI systems. LLMs / transformer-based AI will never get there. This recent paper seems to confirm this: https://arxiv.org/abs/2410.05229
Markus Heinsohn
2024-10-14 08:53:48 +0000 UTC"so laughter and empathy might be the last true universal currency" <3
Christopher Pollin
2024-10-14 05:19:24 +0000 UTCI can't wait to be perma drunk without the negative side effects
Jackson Matysik
2024-10-14 03:42:38 +0000 UTCDirect democracy enabled by personal AI agents could be the greatest reformation of governance in history. Time and intelligence are constraints to democracy as much as any other field. If we can all spawn instances of our perspectives, and have them debate each other and vote, we can resolve all of the governance and peace issues he outlines.
Poss
2024-10-14 00:25:41 +0000 UTCThe key takeway I have from this video is that international competition and financial competion will inevitably drive the decision making around how we develop and how we use AI. This is the way the world currently acts at the global scale.
Barnaby Golden
2024-10-13 23:12:04 +0000 UTCCards on the table: I am a capital D Doomer. That being said: I think that the most impressive application of AI so far is Alphafold, which could in principal, trim the fat, so to speak, from a pool of experimental ideas. That could give you more "intelligent" shots on goal but you still have to deal with the fact that biological systems are unimaginably complicated. I see it all the time where you try something that you think is a slam dunk and it just doesn't work at all. Sometimes you can figure out why based on current knowledge and sometimes you can't. The problem with biology is not an intelligence problem, its a measurement problem. And by that I mean that every biological measurement we make takes hours to days and that is plenty of time for the system to change. When you stick a thermometer into a glass of water the temperature of the glass of the thermometer changes the temperature of the water. We assume that its negligible and we move on. But when someone with ALS dies after slowly suffocating for days and we cut open their brain 30 hours after that, we can't be sure that what we see has any relevance to what caused the disease. "Just cut them open while theyre alive", setting aside the ethical problems with that, you have to deal with the fact that the body mounts an enormous response to traumatic stimuli like that. All of that changes the environment which you are trying to measure, rendering the values suspect. We genetically modify mice to have the same genetic problems that we have, we observe the same disease phenotype in the mice, we develop a drug that cures the mice, and it fails in humans. This happens all the time. One subtle change makes everything null. As far as the price question: Rituximab was released in 1998 and is still $1,000 for 10mg. I know for a fact that you can make anti CD20 monoclonal antibodies for dollars per liter so I don't think that any technological progress will make drugs cheaper at all. Onasemnogene abeparvovec is a genetic treatment for spinal muscular atrophy which probably wasn't developed with what we would call "AI" but certainly would be impossible without computer analysis. It costs $2.125 Million per treatment. I'm not saying its not worth that much, I'm just saying that its not cheap and it would have been impossible to do by hand. As a final thought I really really hope my pessimism is misplaced and that everything will be ok.
theheatdeathiscoming
2024-10-13 23:10:18 +0000 UTC"One area humans are likely to maintain a relative (or even absolute) advantage for a significant time is the physical world. Thus, I think that the human economy may continue to make sense even a little past the point where we reach “a country of geniuses in a datacenter” It seems like Amodei believes humanoid robots won't be perfected/economically viable within the 10 year scope of the essay. I'm a little surprised to hear that but I appreciate his perspective on that. It's personally something I am greatly anticipating.
marvin
2024-10-13 22:59:08 +0000 UTCGenuine question, what are your thoughts on speeding up experimentation (and bringing the costs down) by using AI controlled robots, like they’ve started doing in metallurgy and chemistry? https://www.sciencedaily.com/releases/2024/01/240125145938.htm
Erik
2024-10-13 22:40:02 +0000 UTCTo be frank, I have believed in a takeoff of this speed and magnitude since GPT4 arrived. I have been struggling to explain to people how vast and quick the changes will be and this essay has strongly validated my feelings. Even his specifics are almost all things I have hypothesized over the last couple years. He noticeably hasn't included and predictions on AR/VR and household robots, nor self driving cars or personalized entertainment. I think those things will make the future so much more fun than the simple "we'll cure cancer and governance" part. Of course, we will get it all if we're lucky and careful. Anyway, I'm just excited to watch the journey and am spending my next couple years (as I have my last 5) working on my mental and emotional health, and secondarily my physical health. I don't think I will see much return on investment into career or skills as the last thing tech can do for me will be to improve my brain function and the first thing will be to replace my labour.
John Merkowsky
2024-10-13 20:43:25 +0000 UTCIt's so scary to think that AI can find ways to alter humans in a chemical way, in so many ways, and so much quicker than humans. It's amazing to think of all the progress in health that can come out of this, and in another way, it's really scary to think it can be put in wrong hands.
Tiky
2024-10-13 20:16:39 +0000 UTCThis was a great watch, thank you for this. The problem I have with non-instantaneous takeoffs like the one described here is that many seem to assume that we'll get to a point where we have beings more intelligent than even the smartest of humans after which.... they'll just stop developing? AI research is not bounded by problems like having to wait for cells to multiply or having to run experiments back to back, the only limit to it after we have a digital genius is the compute available that it has to 1) run multiple copies of itself and 2) to run experiments. If we have a country full of nobel price winners working towards the singular goal of building an ever greater AI, how long does it realistically take for them to recursively self improve into the most efficient intelligence form possible on a digital platform? And if not long, I'd say that this kind of "machine god" would then render pretty much all limits we can think of obsolete. If we have a machine god that has the knowledge of every single detail of this world that the internet contains absorbed into its system, available to it as easily and intuitively as 2+2=4 for us, then I find it hard to believe that things could really take that long. I feel like it could form nearly certain hypotheses about pretty much anything that matters to us and is not completely alien, then simply run some verifying experiments in parallel to prove them if necessary. So all in all, I think that before solving anything else, the AI could use all resources to self improve as far as the laws of physics allow it, and only then start solving other problems. If you had the choice of gaining 10 iq points per day or starting to solve some problems now rather than later, which would you pick?
zero
2024-10-13 20:09:20 +0000 UTCI promise that intelligence is not the bottleneck in biology. This essay really goes a long way to show that he has no idea what he's talking about with regards to bio. The fact that he thinks we'd see 1000 years or progress in 10 years is idiotic. There are plenty of diseases where if you snapped your fingers and magically stopped all disease progression you wouldn't be able to tell for 5 years or more. Additionally the vast majority of diseases are rare and largely understood. I don't think an AI is going to run full body simulations to discover the root cause of PSP or something like that. I see this all the time: Math, Physics and CS people tend to think that biology is really easy because undergrad biology is just memorizing bullshit. Thats not what it is in real life. In real life you'll perform the same experiment again and again on clones of the same organism and get different results each time because someone sneezed two rooms over. Additionally I don't know a single biologist who isn't sitting on dozens of experiments that they'd like to run but don't have the time or more importantly money. Calibration experiments alone often cost thousands of dollars. So yeah the bottleneck in biology isn't intelligence. Its just time and money.
theheatdeathiscoming
2024-10-13 19:58:02 +0000 UTCI still get people asking me why I think AGI is here. Lol.
David Shapiro
2024-10-13 19:35:24 +0000 UTCJust finished reading it. Took me a few sittings. I find Amodei’s perspective, approach and language very inspiring. And while that may be naive, he doesn’t sound like a guy who’s in it for the profit.
Nico Appel
2024-10-13 19:31:01 +0000 UTC
