~www_lesswrong_com | Bookmarks (664)

Published on October 17, 2024 3:10 PM GMTDario Amodei is thinking about the potential. The result is a mostly good essay called Machines of Loving Grace, outlining what can be done with ‘powerful AI’ if we had years of what was otherwise relative normality to exploit it in several key domains, and we avoided negative outcomes and solved the control and alignment problems. As...

Published on October 17, 2024 2:23 PM GMTYour mind is a prediction machine, constantly trying to anticipate the world around you and altering its forecasts based on new information. It’s always doing this as a background process. But what would happen if you deliberately trained this skill? Could you get better at predicting your projects, your life, and the future?Sadly, I don’t have a...

Published on October 17, 2024 2:00 PM GMTOf all the cognitive tools our ancestors left us, what’s best? Society seems to think pretty highly of arithmetic. It’s one of the first things we learn as children. So I think it’s weird that only a tiny percentage of people seem to know how to actually use arithmetic. Or maybe even understand what arithmetic is for....

Published on October 17, 2024 1:18 PM GMTAuthors: Federico Adolfi, Martina G. Vilas, Todd Wareham.Abstract:Many proposed applications of neural networks in machine learning, cognitive/brain science, and society hinge on the feasibility of inner interpretability via circuit discovery. This calls for empirical and theoretical explorations of viable algorithmic options. Despite advances in the design and testing of heuristics, there are concerns about their scalability and...

Published on October 17, 2024 5:14 AM GMTIn this episode of the Making Sense podcast with Sam Harris, Barton Gellman from The Brennan Center For Justice discusses how he "organized five nonpartisan tabletop exercises premised on an authoritarian candidate winning the presidency to test the resilience of democratic institutions"."The 175 participants across five exercises were Republicans, Democrats, and independents; liberals, conservatives, and centrists. They...

Published on October 17, 2024 3:37 AM GMTA friend of mine thinks that RL is a dead end: LLMs are much better at problem solving, exploration, and exploitation than any RL algorithm. And I agree that LLMs are better than RL on RL's tasks: companies even have LLMs controlling robots nowadays.The part where we disagree is that I see RL as the step that...

Published on October 16, 2024 11:24 PM GMTFor the next Cognitive Bootcamp, I wanted to experiment with a format that is a) more explicitly “intense”b) a bit more opinionated on how people spend their timewhile c) still flexible enough to deal with the fact that people need different things and being too intense can be badThe next workshop is Friday, Oct 25 at 4pm thru...

Published on October 16, 2024 9:27 PM GMTThe amount of effort is not proportional to the result. One lucid dream (LD) can take hours or even dozens of hours of effort. On average, a practitioner experiences several dozen LDs in their lifetime before quitting. If they don't quit, they dedicate their entire life to it, day and night, trying endless techniques, practicing reality checks,...

Published on October 16, 2024 7:26 PM GMTReading guidelines: If you are short on time, just read the section “The importance of quantitative risk tolerance & how to turn it into actionable signals”Tl;dr: We have recently published an AI risk management framework. This framework draws from both existing risk management approaches and AI risk management practices. We then adapted it into a rating system...

Published on October 15, 2024 6:25 PM GMTThis post was written as part of the summer 2024 cohort of the ML Alignment & Theory Scholars program, under the mentorship of Marius Hobbhahn.AbstractAs model-written evals (MWEs) become more widely used in AI benchmarking, we need scalable approaches to assessing the quality of the eval questions. Upon examination of an existing benchmark dataset (Perez et al., 2022),...

Published on October 15, 2024 4:46 PM GMTToday we are publishing a significant update to our Responsible Scaling Policy (RSP), the risk governance framework we use to mitigate potential catastrophic risks from frontier AI systems. This update introduces a more flexible and nuanced approach to assessing and managing AI risks while maintaining our commitment not to train or deploy models unless we have implemented...

Published on October 15, 2024 3:09 PM GMTAlright, I have a question stemming from TurnTrout's post on Reward is not the optimization target, where he argues that the premises that are required to get to the conclusion of reward being the optimization target are so narrowly applicable as to not apply to future RL AIs as they gain more and more power:https://www.lesswrong.com/posts/pdaGN6pQyQarFHXF4/reward-is-not-the-optimization-target#When_is_reward_the_optimization_target_of_the_agent_But @gwern argued...

Published on October 15, 2024 2:38 PM GMTWhile you can make a lot of progress in evals with tinkering and paying little attention to the literature, we found that various other papers have saved us many months of research effort. The Apollo Research evals team thus compiled a list of what we felt were important evals-related papers. We likely missed some relevant papers, and...

Published on October 15, 2024 2:25 PM GMTI am actively editing this post. Consider reading it in a couple hours; maybe avoid sharing it until it's done.New canonical linkpost: https://www.lesswrong.com/posts/Q7caj7emnwWBxLECF/anthropic-s-updated-responsible-scaling-policy.I haven't yet formed an opinion on the key questions, including are the thresholds and mitigations reasonable and adequate.Anthropic's first update to its RSP is here at last.Today we are publishing a significant update to...

Published on October 15, 2024 1:31 PM GMT5.1 Post summary / Table of contentsThis is the 5th of a series of 8 blog posts, which I’m serializing weekly. (Or email or DM me if you want to read the whole thing right now.)Dissociative Identity Disorder (DID) (previously known as “Multiple Personality Disorder”) involves a person having multiple “alters” (alternate identities), with different preferences and (in some...

Published on October 15, 2024 1:20 PM GMTPrevious Economics Roundups: #1, #2, #3 Fun With Campaign Proposals (1) Since this section discusses various campaign proposals, I’ll reiterate: I could not be happier with my decision not to cover the election outside of the particular areas that I already cover. I have zero intention of telling anyone who to vote for. That’s for you to...

Published on October 15, 2024 12:41 PM GMTIf so, who are they? Link: https://yourbias.is/ At a gloss the material looks really polished and topical. Discuss

Published on October 15, 2024 11:42 AM GMTThere’s a class of problems broadly known as inverse problems. Wikipedia explains them as follows:An inverse problem in science is the process of calculating from a set of observations the causal factors that produced them. [...] It is called an inverse problem because it starts with the effects and then calculates the causes. It is the inverse of a...

Published on October 15, 2024 9:21 AM GMTAuthors: Tianhao Wu, Janice Lan, Weizhe Yuan, Jiantao Jiao, Jason Weston, Sainbayar Sukhbaatar.Summary thread: https://x.com/jaseweston/status/1846011492245672043.Abstract:LLMs are typically trained to answer user questions or follow instructions similarly to how human experts respond. However, in the standard alignment framework they lack the basic ability of explicit thinking before answering. Thinking is important for complex questions that require reasoning and...

Published on October 13, 2024 5:00 PM GMTAs humanity gets closer to Artificial General Intelligence (AGI), a new geopolitical strategy is gaining traction in US and allied circles, in the NatSec, AI safety and tech communities. Anthropic CEO Dario Amodei and RAND Corporation call it the “entente”, while others privately refer to it as “hegemony" or “crush China”. I will argue that, irrespective of one's ethical...

Published on October 13, 2024 2:00 PM GMT In general I'd like to see a lot more of people writing about their failures in addition to their successes. If a bunch of people all try a thing and have mixed results, and only the people with good results write about it, people who don't know about this selection bias or don't realize its extent...

Published on October 13, 2024 3:01 AM GMTThis is a rough outline of my philosophical framework. Which gives me a context for all of my experiences and knowledge. Certain KnowledgeWe cannot know much for certain. I am a ultimate skeptic however I agree with Rene Descartes so far as I can certainly trust in my own existence. My conscious experience in this moment is guaranteed....

Published on October 12, 2024 5:36 PM GMTTL;DR: In this post I discuss a recently proposed on-chip compute governance mechanism, a delay based location verification mechanism. Then I demonstrate feasibility and applicability of the mechanism through a few exercises and identify a potential issue with frequent false positives. Finally I propose a potential solution for the identified issue.Technical governance approaches will be critical for...