~www_lesswrong_com | Bookmarks (664)

Published on October 23, 2024 5:39 AM GMTI've written a lot of words—hundreds of blog posts, thousands of comments, tens of thousands of emails, and hundreds of thousands of short messages. I'm even written most of a book! In total, between personal and professional writing, I estimate to have put north of 3 million words into text. So it may come as a surprise...

Published on October 22, 2024 9:57 PM GMTRecently the whole "if your p(doom) is high, you should short the market" thing has been going around. Let's say the market prices in a 0.1% chance of extinction per century, and you think there's a 50% chance we're dead in the next 25 years. This is about 12 bits of evidence. How much profit is it...

Published on October 22, 2024 8:57 PM GMTThanks to Holden Karnofsky, David Duvenaud, and Kate Woolverton for useful discussions and feedback.Following up on our recent “Sabotage Evaluations for Frontier Models” paper, I wanted to share more of my personal thoughts on why I think catastrophic sabotage is important and why I care about it as a threat model. Note that this isn’t in any...

Published on October 22, 2024 6:20 PM GMT~5 months I formally quit EA (formally here means “I made an announcement on Facebook”). My friend Timothy was very curious as to why; I felt my reasons applied to him as well. This disagreement eventually led to a podcast episode, where he and I try convince each other to change sides on Effective Altruism- he tries...

Published on October 21, 2024 5:56 PM GMTHere I define autonomy as not having your insides controlled by your outsides. Complete autonomy is impossible, of course. This is the lim→∞.For example, imagine a bacterium. It has insides: its guts. It has outsides: its environment.And in between, it has a membrane. The membrane is the thing that attempts to control what comes in and what goes out. For example, pores can guide nutrients...

Published on October 21, 2024 5:10 PM GMTI'm an advocate of something known as sortition. The premise is simple. Choose people at random, to serve a finite term, in some decision making capacity. Pay them to be there. Sounds ridiculous, right?  How could we possibly trust ignorant, stupid, normal people to make good decisions? What would this look like? Why would this be better than...

Published on October 21, 2024 3:52 PM GMTNote: this essay is co-written with Eryney Marrogi, who helped seed the initial idea and edited this piece a fair bit. On a related note, I’m helping him run an NYC meetup event on Wednesday, November 20th in Williamsburg, you should sign up here to come! If you like biology, ML, or human connection, I highly recommend...

Published on October 21, 2024 2:50 PM GMTSummary: If we validate automated alignment research through empirical testing, the safety assurance work will still need to be done by humans, and will be similar to that needed for human-written alignment algorithms.Three levels of automated AI safetyAutomating AI safety means developing some algorithm which takes in data and outputs safe, highly-capable AI systems. Let’s imagine three...

Published on October 21, 2024 2:00 PM GMTThis post is incomplete. I'm publishing it because it might be helpful for some readers anyway. A good version of this post would be more detailed: for each proposed action, explain motivation and high-level goal, explain problem to solve, respond to obvious questions/objections, and lay out costs involved and costs worth accepting.These are the actions I think...

Published on October 21, 2024 1:57 PM GMTA sarcastic yet genuine question. Even in light of this discussion and the parallel one on the EA Forum, I can't make out at what conditions 80k would cease to (implicitly) endorse OpenAI.Discuss

Published on October 21, 2024 1:55 PM GMT 1. 4.4% of the US federal budget went into the space race at its peak.This was surprising to me, until a friend pointed out that landing rockets on specific parts of the moon requires very similar technology to landing rockets in soviet cities.[1]I wonder how much more enthusiastic the scientists working on Apollo were, with the convenient motivating...

Published on October 21, 2024 8:43 AM GMTLast April, AI Safety Tokyo and Noeon Research (in collaboration with Reaktor Japan, AI Alignment Network and AI Industry Foundation) hosted TAIS 2024, an AI safety conference in Tokyo, Japan. You can read more about that conference and how well it went here.We are running another conference next April (TAIS 2025), and have just put out our call for papers....

Published on October 21, 2024 1:26 AM GMTPregnancy is most stressful at the beginning and at the end. First you try to conceive, and spend half of each month superimposed between two very different realities. Then there’s morning sickness, and you worry about miscarriage, which is far more likely early on.Near the end, you can be pretty confident you’re having a baby. But there’s...

Published on October 19, 2024 6:11 AM GMTWhat actually bad outcome has "ethics-based" AI Alignment prevented in the present or near-past? By "ethics-based" AI Alignment I mean optimization directed at LLM-derived AIs that intends to make them safer, more ethical, harmless, etc. Not future AIs, AIs that already exist. What bad thing would have happened if they hadn't been RLHF'd and given restrictive system...

Published on October 19, 2024 3:58 AM GMTSimulating Political AlignmentThis methodology concerns a simulation tool which has been developed to model how beliefs, that are not directly related, end up correlated in political identities. It models the transmission of beliefs between nodes on a static hexagonal grid, based on a valence matrix. This methodology allows the user to observe the spread of ideas and...

Published on October 19, 2024 2:19 AM GMTI see widespread dismissal of AI capabilities. This slows down the productivity gains from AI, and is a major contributor to disagreements about the risks of AI. It reminds me of prejudice against various types of biological minds. I will try to minimize the moralizing about fairness in this post, and focus more on selfish reasons to...

Published on October 19, 2024 2:00 AM GMT While this post touches on biosecurity it's a personal post and I'm not speaking for my employer If you want to prevent airborne spread of diseases you have a few options: Filter breath (masks, PAPRs) Replace the air (ventilation) Clean the air (filters, UV light) Masks, fans, and air filters are widely available, but what about...

Published on October 18, 2024 11:39 PM GMTPreamble My alignment proposal involves aligning an encoding of human-friendly values and then turning on a self-improving AGI with that encoding as its target. Obviously this involve "aligning an encoding of human-friendly values" and also "turning on a self-improving AGI with a specific target", two things we currently do not know how to do... As expected, this...

Published on October 18, 2024 10:33 PM GMTThis is a linkpost for a new research paper from the Alignment Evaluations team at Anthropic and other researchers, introducing a new suite of evaluations of models' abilities to undermine measurement, oversight, and decision-making. Paper link.Abstract:Sufficiently capable models could subvert human oversight and decision-making in important contexts. For example, in the context of AI development, models could...

Published on October 18, 2024 10:02 PM GMTThis is an entry in the 'Dungeons & Data Science' series, a set of puzzles where players are given a dataset to analyze and an objective to pursue using information from that dataset.Estimated Complexity: 4/5  (this is a guess, I will update based on feedback/seeing how the scenario goes)STORYThe Demon King rises in his distant Demon Castle....

Published on October 18, 2024 9:47 PM GMTI came up with a technique that I have derived much benefit from; what kids call a "lifehack" these days. I have done this for many months, found it extremely useful and the usefulness has been rising fairly steadily as I got better at it. All the friends I could get to try it gave very good...

Published on October 18, 2024 6:13 PM GMTTLDR: In response to Leopold Aschenbrenner’s ‘Situational Awareness’ and its accelerationist national ambitions, we argue against the claim that artificial superintelligence will inevitably be weaponised and turn its country of origin into an untouchable hegemony. Not only do we see this narrative as extremely dangerous, but also expect that the grandest AI challenges call for global coordination...