$7 Trillion, a Bioweapon and a Nuke In Space - Under-the-Radar AI Safety Papers
Added 2024-02-22 15:38:51 +0000 UTCEverything you missed in the world of AI threats because of Sora and Gemini. From Compute Overhang @Sama, to a laudable Bioweapon study from OpenAI, and from state-actors using GPT-4 to the future of warfare.
Goody-2: https://www.goody2.ai/chat
State-Actors Use AI: https://openai.com/blog/disrupting-malicious-uses-of-ai-by-state-affiliated-threat-actors
Microsoft Update: https://www.microsoft.com/en-us/security/blog/2024/02/14/staying-ahead-of-threat-actors-in-the-age-of-ai/
LLMs Can Autonomously Hack: https://arxiv.org/pdf/2402.06664v1.pdf
Phishing Surge: https://www.cnbc.com/2023/11/28/ai-like-chatgpt-is-creating-huge-increase-in-malicious-phishing-email.html#:~:text=Technology%20Executive%20Council-,AI%20tools%20such%20as%20ChatGPT%20are%20generating,increase%20in%20malicious%20phishing%20emails&text=Since%20the%20fourth%20quarter%20of,report%20by%20cybersecurity%20firm%20SlashNext.
Insider Risk Investigator: https://openai.com/careers/insider-risk-investigator
Biological Threat Creation: https://openai.com/research/building-an-early-warning-system-for-llm-aided-biological-threat-creation
2-3 Years Amodei: https://www.theregister.com/2023/07/28/ai_senate_bioweapon/
Anthropic Read Teaming: https://www.anthropic.com/news/frontier-threats-red-teaming-for-ai-safety
RAND Report: https://www.rand.org/pubs/research_reports/RRA2977-2.html
Dual Use: https://arxiv.org/ftp/arxiv/papers/2306/2306.03809.pdf
Release the Weights?: https://arxiv.org/ftp/arxiv/papers/2310/2310.18233.pdf
Emergent Autonomous Abilities: https://arxiv.org/ftp/arxiv/papers/2304/2304.05332.pdf
Nuke in Space: https://www.bloomberg.com/news/articles/2024-02-20/us-tells-allies-russia-may-put-anti-satellite-nuclear-weapon-in-space-this-year
AI Drone Swarm: https://www.bloomberg.com/news/articles/2024-02-17/us-uk-may-arm-ukraine-with-ai-enabled-drones-to-target-russian-positions
Huang says $2 Trillion: https://www.bloomberg.com/news/articles/2024-02-12/developing-ai-shouldn-t-cost-7-trillion-nvidia-ceo-jensen-huang-says?srnd=technology-ai
Altman says $7 trillion all-in: https://www.theinformation.com/articles/no-sam-altman-isnt-raising-trillions-of-dollars-for-chips?rc=sy0ihq
Or 8... https://twitter.com/sama/status/1758347811786281355
https://twitter.com/sama/status/1739360234832052641
Compute Overhang: https://openai.com/blog/planning-for-agi-and-beyond
Comments
I'm way less concerned about the bioweapons than I am about the spear fishing, propaganda and pig butchering. The most powerful global threat actors are state affiliated or organised crime, and neither of those groups want a biowar. They definitely do want to steal money, spread lies, totalitarianism and corruption. The end state outcome of that goal unmitigated is the breakdown of society as we know it. That breakdown magnifies underlying drivers for terrorism, including bioterrorism. Overall risk assessment is not just a function of impact, but of likelihood. AI-enabled cyber comms are already happening, and we already see evidence of societal disruption in the order of tens of billions in scams, and measurable impact on elections. That's a 100% chance of occurrence and a 100% chance of impact. That's light years ahead of the current threat profile for bioweapons or hostile AGI
Poss
2024-02-23 22:43:26 +0000 UTCGreat video! Thank you :D
Jakub Nekvasil
2024-02-23 08:24:54 +0000 UTCGreat overview Philip, this is definitely an example of an area that I want to be informed on, but don't have the time for to research myself. So it's awesome that you did!
Erik
2024-02-23 00:59:47 +0000 UTCcovid was made in a lab my friend
Dane Wagenhoffer
2024-02-22 21:20:57 +0000 UTCWhat a time to be alive.
Jonathan Kirk
2024-02-22 19:15:27 +0000 UTCIn recent years there has been a huge progress in tools for synthesize biological material. Even if risk of successful bioterrorism act is small - the potential casualties are enormous (COVID was a relatively mild virus comparing to what is possible). This podcast with the inventor of artificial gene drive is very informative in this matter: https://80000hours.org/podcast/episodes/kevin-esvelt-stealth-wildfire-pandemics/ (though other experts might have different opinions about plausibility of these scenarios)
Jan Matusiewicz
2024-02-22 17:52:08 +0000 UTCThank you for the great video. I have two reflections: 1. it is great that measures are being taken to decrease chances of using ChatGPT for bio-terrorism. But why is general public ignoring the fact that wet markets still operate in some countries (https://www.telegraph.co.uk/global-health/science-and-disease/why-wet-markets-will-never-close-despite-global-threat-human/) ? Humanity may sooner or later have another 7m deaths pandemic due to negligence. 2. Idea of being hunted down by a killer drone is surely terrifying. But so is crawling through a minefield under artillery fire and that's what Ukrainians soldiers have to do. I don't quite see why it would make a difference for a soldier if a drone that will kill him is controlled by a human operator or works autonomously to prevent jamming. I don't think it is adequate to compare drone swarm to chemical weapon. If anything weapon that kills slowly and indiscriminately is even more cruel that one that kills quickly and targets soldiers only. Also arm race between Russian and Ukrainian drones is already ongoing, helping Ukraine build better drones will not change that.
Jan Matusiewicz
2024-02-22 17:45:49 +0000 UTC"Have a wonderful day" is a somewhat jarring way to close that particular segment! About as uplifting as a Yudkowski tweet ;-)
Mike Pemberton
2024-02-22 17:41:20 +0000 UTCI don't quite see the bio weapons threat. We already have malicious actors, and they are not dumb, they have smart people and scientists. They don't need ChatGPT. Maybe I'm ignorant, but I feel like actually building bio weapons or creating new diseases in a lab is way more difficult than just study a bit, order some chemicals and brew everything in a cooking pot. Otherwise we would see this happening all the time. Similar to building a nuclear bomb, which every physics student in the world learns how to do, it's probably just very hard to get all the machinery and ingredients to make a bio weapon in a controlled way, not accidentally killing yourself or your own people. And even if you created a new strand of Covid or so, it would most likely just immediately die off simply because it was created artificially and not put to test by the mechanism of evolution.
Phillip Lakaschus
2024-02-22 17:11:50 +0000 UTCThe best use (in this case) of agents is to serve as information concierges e.g. an intermediary service that interacts with the internet for you.
David Shapiro
2024-02-22 16:04:00 +0000 UTC
