~www_lesswrong_com | Bookmarks (688)

Published on January 18, 2025 3:32 PM GMTWhen learning about classic utilitarianism (approximately, the quest to maximize everyone's expected well-being), I struggle because much of my well-being seems internal. If happiness or misery are significantly influenced by our internal processing of events, then how does this affect utilitarianism and its practical application?I'll start with a few examples:When I was a child, my family moved...

Published on January 18, 2025 3:39 PM GMTHi, everyone. I'm not sure if my post is well-written, but I think LW might be the only right place to have this discussion. Feel free to suggest changes.AGI may arrive soon and it is possible that it would kill us all. This does not bother me that much, as dying happens to people all the time.But...

Published on January 18, 2025 4:00 PM GMT In general, you're not supposed to wear a beard with a respirator mask (N95, P100, etc), at least not in a way where you have facial hair under the seal: But how much worse is the fit? A P100 with a beard is going to filter less well than a P100 without a beard, but does...

Published on January 18, 2025 3:29 PM GMTConfusion is a felt sense; a bodily sensation you can pay attention to and notice! So here's my question for you:What's this Song about?Think About Things by Daði FreyYoutube Link, Spotify LinkFull Lyrics:Baby I can't wait to knowBelieve me I'll always be there soThough I know I love youI find it hard to see how you feel...

Published on January 18, 2025 3:10 PM GMTWe’re developing an AI-enabled wargaming-tool, grim, to significantly scale up the number of catastrophic scenarios that concerned organizations can explore and to improve emergency response capabilities of, at least, Sentinel.Table of ContentsHow AI Improves on the State of the ArtImplementation Details, Limitations, and ImprovementsLearnings So FarGet Involved!How AI Improves on the State of the ArtIn a wargame,...

Published on January 18, 2025 12:43 PM GMTepistemic status: I know next to nothing about evolution, development psychology, AI, alignment. Anyway, I think the topic is important, and I should do my, however small part, in trying to think seriously for 5 minutes about it. So here's what I thinkHow come, that I am aligned? Somehow neocortex plays along with older parts of the...

Published on January 18, 2025 9:20 AM GMTI think a lot of people have heard so much about internalized prejudice and bias that they think they should ignore any bad vibes they get about a person that they can’t rationally explain. But if a person gives you a bad feeling, don’t ignore that. Both I and several others who I know have generally come...

Published on January 18, 2025 3:54 AM GMTIntroduction: Why QFT?In a previous post, Lauren offered a take on why a physics way of thinking is so successful at understanding AI systems. In this post, we look in more detail at the potential of Quantum field theory (QFT) to be expanded into a more comprehensive framework for this purpose. Interest in this area has been...

Published on January 17, 2025 9:24 PM GMTCross-posted from SubstackFeeling intellectually understimulated, I've begun working my way through Max Lamparth's CS120 - Introduction to AI Safety. I'm going to use this Substack as a kind of open journaling practice to record my observations on the ideas presented, both in the lectures and in the readings.The reading for the first week is Dobbe, Gilbert, and...

Published on January 17, 2025 9:45 PM GMTInner Alignment is the problem of ensuring mesa-optimizers (i.e. when a trained ML system is itself an optimizer) are aligned with the objective function of the training process.Inner alignment asks the question: How can we robustly aim our AI optimizers at any objective function at all?As an example, evolution is an optimization force that itself 'designed' optimizers...

Published on January 16, 2025 6:03 PM GMTThis is a linkpost for How should we analyse survey forecasts of AI timelines? by Tom Adamczewski, which was published on 16 December 2024[1]. Below are some quotes from Tom's post, and a bet I would be happy to make with people whose AI timelines are much shorter than those of the median AI expert.How should we...

Published on January 16, 2025 5:20 PM GMTSummaryCan LLMs science? The answer to this question can tell us important things about timelines to AGI. In this small pilot experiment, we test frontier LLMs on their ability to perform a minimal version of scientific research, where they must discover a hidden rule about lists of integers by iteratively generating and testing hypotheses. Results are ambiguous:...

Published on January 16, 2025 5:19 PM GMTI would like to thank Professor Vincent Conitzer, Caspar Oesterheld, Bernardo Subercaseaux, Matan Shtepel, and Robert Trosten for many excellent conversations and insights. All mistakes are my own.I think that there's a fundamental connection between AI Safety via Debate and Guaranteed Safe AI via topology. After thinking about AI Safety via Debate for nearly two years, this...

Published on January 16, 2025 2:20 PM GMTThe fun, as it were, is presumably about to begin. And the break was fun while it lasted. Biden went out with an AI bang. His farewell address warns of a ‘Tech-Industrial Complex’ and calls AI the most important technology of all time. And there was not one but two AI-related everything bagel concrete actions proposed –...

Published on January 16, 2025 1:55 PM GMTNB this dialogue occurred at the very end of 2023, and for various reasons is only being published ~a year later! Keep this in mind while reading.Key topics coveredconsistent NN-internal deceptive planning (search: goblin, fae, homunc, inner)making AI/agents from foundation models (search: scaffold, condition)planning (search: planning, simulator, consequential)standards of evidence (search: fanfic, theory, Shrek, analog, logic)elicitation gaps...

Published on January 16, 2025 11:38 AM GMTWeirdML websiteRelated posts:How good are LLMs at doing ML on an unknown dataset?o1-preview is pretty good at doing ML on an unknown datasetIntroductionHow good are Large Language Models (LLMs) at doing machine learning on novel datasets? The WeirdML benchmark presents LLMs with weird and unusual machine learning tasks, designed to require careful thinking and actual understanding to...

Published on January 16, 2025 10:51 AM GMTFrom the earliest days of evolutionary thinking, we’ve used metaphors to understand how life changes over time. One of the most enduring is the image of a vast “fitness landscape” with countless peaks and valleys, each corresponding to different levels of survival and reproduction. This landscape is a way of imagining how living things (the “replicators”) search...

Published on January 16, 2025 8:53 AM GMTLearning quantum mechanics involves two things:learning the fundamentals, the nature of the stuff of which Nature is made; andloads of universe-imposed incidental complexity, spinors and vector calculus and Lie algebras and Hilbert spaces, in order to be able to effectively apply the fundamentals to the real world.Now, me, personally, I really like the feeling of understanding how...

Published on January 16, 2025 8:04 AM GMTToday's "nanowrimo" post is a fun longform introduction to permanents and their properties, organized in the way I wish it had been explained to me. Note before going on: this is not my field, and this post is even more likely than usual to contain incorrect statements or bugs.Recall that given an n×n.mjx-chtml {display: inline-block; line-height: 0; text-indent:...

Published on January 16, 2025 2:14 AM GMT(Explanation. Also I have no reason to think they hate me.)Do not use the original TruthfulQA multiple-choice or the HaluEval benchmark. We show that a simple decision tree can theoretically game multiple-choice TruthfulQA to 79.6% accuracy—even while hiding the question being asked! In response, the TruthfulQA authors created a new multiple-choice condition which avoids the vulnerabilities we...

Published on January 15, 2025 6:16 PM GMTApplications are now open for the Cooperative AI Summer School, which will take place from 9th to 13th July 2025 in Marlow, near London! Designed for students and early-career professionals in AI, computer science, and related disciplines—such as sociology and economics—the summer school offers a firm grounding in the emerging field of cooperative AI.Confirmed speakers for this year include:Michael Wellman (University...

Published on January 15, 2025 6:00 PM GMTI'm collecting (x-risk-relevant) safety research from frontier AI companies published in 2023 and 2024: https://docs.google.com/spreadsheets/d/10_dzImDvHq7eEag6paK6AmIdAGMBOA7yXUvumODhZ5U/edit?usp=sharing.I was planning to get AI safety researchers to score each of the papers, so that we could compare the labs on quality-adjusted safety research output. I'm giving up on this for now, largely because I expect to struggle to find scorers. Let...

Published on January 15, 2025 3:55 PM GMTSome fraction of the time, LLMs naturally go on existential rants. My best guess is that, just as people can flip into a context where we do that, so can LLMs. With the result that the LLM certainly sounds like it is suffering, even if we discount the possibility that it actually is. Which raised a question....

Published on January 15, 2025 2:50 PM GMTSocial networks have shaped me since a young age. Growing up at the beginning of the millennium, I used to spend my time in phpBB and vBulletin forums. There, I befriended internet strangers, started my way into graphic design, and learned about torrents. Forums were my favorite third places—little corners on the web where I felt a...