~www_lesswrong_com | Bookmarks (669)

Published on November 4, 2024 6:16 PM GMT8.1 Post summary / Table of contentsThis is the final post of the Intuitive Self-Models series.One-paragraph tl;dr: This post is, in a sense, the flip side of Post 3. Post 3 centered around the suite of intuitions related to free will. What are these intuitions? How did these intuitions wind up in my brain, even when they...

Published on November 4, 2024 5:54 PM GMTIntroduction and summary(This is the third in a series of four posts about how we might solve the alignment problem. See the first post for an introduction to this project and a summary of the posts that have been released thus far.)In the first post in this series, I distinguished between “motivation control” (trying to ensure that a superintelligent...

Published on November 4, 2024 4:00 PM GMTThis is a reference post. It contains no novel facts and almost no novel analysis.The idea of responsible scaling policies is now over a year old. Anthropic, OpenAI, and DeepMind each have something like an RSP, and several other relevant companies have committed to publish RSPs by February.The core of an RSP is a risk assessment plan plus a plan...

Published on November 4, 2024 3:20 PM GMTProponents of spirituality and alternative medicine often use the argument "this has been practiced for 2000 years", with the subtext "therefore it must work". Does this argument have any validity?At first glance I want to reject the argument entirely, but that might be premature. Are there situations where this kind of argument is valid or somewhat valid?I...

Published on November 4, 2024 2:35 PM GMTLooking back from 2041When people in the early 21st Century imagined an AI-empowered economy, they tended to project person-like AI entities doing the work. “There will be demand for agent-like systems,” they argued, “so we’ll see AI labs making agents which can then be deployed to various problems”.We now know that that isn’t how it played out....

Published on November 4, 2024 11:10 AM GMT(This was inspired by a conversation with Alex Altair and other fellows as part of the agent foundations fellowship, funded by the LTFF)(Also: after I had essentially finished this piece, I was pointed toward the post Natural abstractions are observer-dependent which covers a lot of similar ground. I've decided to post this one anyway because it comes...

Published on November 4, 2024 10:15 AM GMTAbstractAs AI closely interacts with human society, it is crucial to ensure that its decision-making is safe, altruistic, and aligned with human ethical and moral values. However, existing research on embedding ethical and moral considerations into AI remains insufficient, and previous external constraints based on principles and rules are inadequate to provide AI with longterm stability and...

Published on November 4, 2024 9:29 AM GMTThis dialogue is still in progress, but due to other commitments we don't have much time to continue it. We think the content is interesting, so we decided to publish it unfinished. We will maybe very slowly continue adding to it in the future, but can't commit to doing so.Jeremy GillenContext: I wrote a shortform about conditioning...

Published on November 4, 2024 2:29 AM GMTI open my eyes and find myself lying on a bed in a hospital room. I blink."Hello", says a middle-aged man with glasses, sitting on a chair by my bed. "You've been out for quite a long while.""Oh no ... is it Friday already? I had that report due -""It's Thursday", the man says."Oh great", I say....

Published on November 3, 2024 11:13 PM GMTThis is a cross-post from my new newsletter, where I intend to post about clinical trials and biotech, and from my personal blog, where I intend to go on posting about all other sorts of topics as well. I also recently went on the Complex Systems podcast, where I discuss these topics with more examples from actual...

Published on November 2, 2024 5:38 PM GMTIn the past 3 and a bit years since I left my successful career as a data scientist and machine learning engineer, I've been applying to various AI alignment/safety positions and getting turned down. Job application rejections generally don't give any feedback, so I'm left rather in the dark about what the reasons might be. I was a...

Published on November 2, 2024 5:25 PM GMTWhat do I want to do?This sounds like a question with an obvious answer, but the more I reflect on it, the more I notice complexity and subtleties.First, there's a difficult tradeoff between what I want locally and globally: I regularly wish to do thing in the moment that would go against my global preferences (eat too...

Published on November 2, 2024 4:18 PM GMTI recently saw a post arguing that top AI labs should shut down. This let me wonder whether the AI Safety community thinks OpenAI is net negative for AI safety. I chose OpenAI because I consider it as the most representative top AI lab (in the sense that, if we ask someone to think about an AI...

Published on November 1, 2024 6:55 PM GMTBharath Puranam (bharath225525@gmail.com)This research blog represents my final project for the AI Safety Fundamentals: AI Alignment Course for June 2024 Cohort.Update: Finally received my certification for the course: AI Safety Fundamentals: Alignment | Bharath PuranamImagine a world where artificial intelligence doesn't just assist in education, but actively shapes how teenagers learn. That's the future we're exploring in...

Published on November 2, 2024 10:22 AM GMTEpistemic status: shower thoughts.I am currently going through the EA Introductory Course and we discussed two arguments against longtermism which I have not seen elsewhere.So goes a thought experiment: imagine you have toxic waste at hand, which you can process right now at the cost of 100 lives, or bury it so it'll have no effect right...

Published on November 2, 2024 3:04 AM GMTIn a few days time, voting will close for millions of Americans in one of the most contentious and globally consequential elections in world history. And while this week’s subject; both-sidesism, is ‘evergreen’ in that the topic will continue to be relevant long into the future, this election has highlighted its significance and introduced some new turns.A...

Published on November 1, 2024 11:53 PM GMTCryptocurrency is terrible.  With a single click of a button, it is possible to accidentally lose all of your funds.  99.9% of all cryptocurrency projects are complete scams (conservative estimate).  Crypto is also tailor-made for ransomware attacks, since it makes it possible to send money in such a way that the receiver has perfect anonymity.typical web3 experienceSimilarly,...

Published on November 1, 2024 11:06 PM GMTMajor scientific institutions talk a big game about innovation, but the reality is that many of the mechanisms designed to ensure quality—peer review, funding decisions, the academic hierarchy—explicitly incentivize incremental rather than revolutionary progress.[1] Thomas Kuhn's now-famous notion of paradigm shifts was pointing at precisely this phenomenon. When scientists work within what Kuhn called "normal science," they're essentially solving low- to...

Published on November 1, 2024 6:56 PM GMTI've been thinking about the set theory multiverse and its philosophical implications, particularly regarding mathematical truth. While I understand the pragmatic benefits of the multiverse view, I'm struggling with its philosophical implications. The multiverse view suggests that statements like the Continuum Hypothesis aren't absolutely true or false, but rather true in some set-theoretic universes and false in...

Published on November 1, 2024 7:23 PM GMTSubhash and Josh are co-first authors. Work done as part of the two week research sprint in Neel Nanda’s MATS streamTLDRWe show that dense probes trained on SAE encodings are competitive with traditional activation probing over 60 diverse binary classification datasetsSpecifically, we find that SAE probes have advantages with:Low data regimes (~ < 100 training examples)Corrupted data (i.e. our...

Published on October 31, 2024 5:25 PM GMTRecently, in a conversation with a coworker, I was trying to describe the rate at which time passed subjectively, with a term that distinguished from the usual objective clock speed constrained only by general relativity and generic human psychology. I ended up saying "meta-rate". That bothered me for reasons I couldn't put a finger on until today.I...

Published on October 31, 2024 5:20 PM GMTDevelopers of frontier AI systems will face increasingly challenging decisions about whether their AI systems are safe enough to develop and deploy. One reason why systems may not be safe is if they engage in scheming. In our new report "Towards evaluations-based safety cases for AI scheming", written in collaboration with researchers from the UK AI Safety...

Published on October 31, 2024 3:00 PM GMTFollowing up on the Biden Executive Order on AI, the White House has now issued an extensive memo outlining its AI strategy. The main focus is on government adaptation and encouraging innovation and competitiveness, but there’s also sections on safety and international governance. Who knows if a week or two from now, after the election, we will...

Published on October 31, 2024 12:01 PM GMTWe (Connor Leahy, Gabriel Alfour, Chris Scammell, Andrea Miotti, Adam Shimi) have just published The Compendium, which brings together in a single place the most important arguments that drive our models of the AGI race, and what we need to do to avoid catastrophe.We felt that something like this has been missing from the AI conversation. Most of...