~www_lesswrong_com | Bookmarks (669)

Published on August 1, 2024 5:33 PM GMTAt my work, we run experiments – we specify some set of input parameters, run some code, and get various metrics as output. Since we run so many of these, it's important for them to be fast and cheap.Recently I was working on an experiment type that took about ~1 hour per run, where the slow part was...

Published on August 1, 2024 3:06 PM GMTDisclaimer: I am outside of the world of international organisations. I am a scientific researcher at university. I am writing this post to open a discussion.IntroductionUN is an international organisation with the following main goals:maintain international peace and securitydevelop friendly relations among nationsstand up for human rightspromote better living standards and social progressHere a more concrete list...

Published on August 1, 2024 5:14 PM GMTTL;DR: Let’s start iterating on experiments that approximate real, society-scale multi-AI deploymentEpistemic status: These ideas seem like my most prominent delta with the average AI Safety researcher, have stood the test of time, and are shared by others I intellectually respect. Please attack them fiercely!Multi-polar risksSome authors have already written about multi-polar AI failure. I especially like how Andrew...

Published on August 1, 2024 5:00 PM GMT I'm agnostic on the existence of dragons. I don't usually talk about this, because people might misinterpret me as actually being a covert dragon-believer, but I wanted to give some background for why I disagree with calls for people to publicly assert the non-existence of dragons. Before I do that, though, it's clear that horrible acts...

Published on August 1, 2024 4:29 PM GMTA couple of months ago I created a meetup group for ACX-adjacent people in Morristown (https://www.meetup.com/morristown-nj-friendly-ambitious-nerds/)About ~7 people have been meeting up weekly and it's been going great. I'm excited to expand the group by hosting an ACX meetup in Morristown!We'll meet at the center of the Green (https://plus.codes/87G7QGW9+RJC)If the weather is good we can hang out...

Published on August 1, 2024 3:17 PM GMTAfter reading another article on IQ, there are a few things that I wish would become common knowledge to increase the quality of the debate. Posting them here:1)There is a difference between an abstract definition of intelligence such that it could also apply to aliens or AIs (something like "an agent able to optimize for outcomes in...

Published on August 1, 2024 1:40 PM GMTGoogle DeepMind got a silver metal at the IMO, only one point short of the gold. That’s really exciting. We continuously have people saying ‘AI progress is stalling, it’s all a bubble’ and things like that, and I always find remarkable how little curiosity or patience such people are willing to exhibit. Meanwhile GPT-4o-Mini seems excellent, OpenAI...

Published on August 1, 2024 10:27 AM GMTSocial anxiety is one hell of a thing. I used to struggle with it a lot — escaping pressure by fleeing to the toilet. I’ve reduced my levels of social anxiety by bashing it over the head with exposure therapy, repeatedly dealing with anxiety-provoking situations until they became manageable.Nowadays, my levels of social anxiety are low enough...

Published on August 1, 2024 4:49 AM GMTThe audio version can be listened to here:In this essay, I will give my views on technology and progress, focusing specifically on space travel, computers and transhumanism.Futurism is not just predictive. It is also normative. We are concerned with what we should do, not just with what will happen. So, I’m going to talk about what I...

Published on August 1, 2024 1:15 AM GMTI did what I think is the largest piece of research on current and aspiring AI safety folk in Australia & New Zealand. I wanted to understand their career barriers so that I could then optimize my organization's tactics to remove them.CaveatsI am not a trained social scientist nor a statistician. There will be errors.I’ve budgeted a few...

Published on July 30, 2024 4:22 PM GMTFigure 1. Image generated by DALL-3 to represent the concept of self-other overlapMany thanks to Bogdan Ionut-Cirstea, Steve Byrnes, Gunnar Zarnacke, Jack Foxabbott and Seong Hah Cho for critical comments and feedback on earlier and ongoing versions of this work.SummaryIn this post, we introduce self-other overlap training: optimizing for similar internal representations when the model reasons about...

Published on July 30, 2024 3:41 PM GMTThis post is an interim progress report on work being conducted as part of Berkeley's Supervised Program for Alignment Research (SPAR).Summary of Key PointsWe test the robustness of an open-source LLM’s (Llama3-8b) ability to recognize its own outputs on a diverse mix of datasets, two different tasks (summarization and continuation), and two different presentation paradigms (paired and...

Published on July 30, 2024 2:57 PM GMTThanks to Zora Che, Michael Chen, Andi Peng, Lev McKinney, Bilal Chughtai, Shashwat Goel, Domenic Rosati, and Rohit Gandikota.TL;DRIn contrast to evaluating AI systems under normal "input-space" attacks, using "generalized," attacks, which allow an attacker to manipulate weights or activations, might be able to help us better evaluate LLMs for risks – even if they are deployed...

Published on July 30, 2024 1:10 PM GMTSome in the tech industry decided now was the time to raise alarm about AB 3211. As Dean Ball points out, there’s a lot of bills out there. One must do triage. Dean Ball: But SB 1047 is far from the only AI bill worth discussing. It’s not even the only one of the dozens of AI...

Published on July 30, 2024 5:36 AM GMTAfter consuming new information, it can be considered:Trivia if I intended to use it solely as entertainment value.Mental Masturbation[1] because while perhaps I enjoyed learning it, I failed to learn something useful that produced a tangible effect.Effective Information which I define as something that had a measurable effect on me.While stimulating myself with new information all day long...

Published on July 30, 2024 12:00 AM GMT A few weeks ago Julia wrote about how we approach kids climbing: The basics: Spot the child if they're doing something where a fall is likely. Don't encourage or help the child to climb something that's beyond their ability to do on their own. If they don't know how to get down, give advice rather than...

Published on July 29, 2024 5:50 PM GMTWelcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required.Listen to the AI Safety Newsletter for free on Spotify or Apple Podcasts.Implications of a Trump administration for AI policyTrump named Ohio Senator J.D. Vance—an AI regulation skeptic—as his pick for vice president. This choice...

Published on July 29, 2024 2:50 PM GMTThis is an informal note on an interpretability illusion I've personally encountered, twice, in two different settingsCausal AnalysisLet's say we have a function f.mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none;...

Published on July 29, 2024 10:28 AM GMTI was thinking about how prompt differs from training data in terms of tokenization. If i am to prompt with "solution:" as opposed to "solution: " it seems like it can influence the result, as in training data last token contain some information about next token. If there is token ": T" but my prompt ended in...

Published on July 29, 2024 9:36 AM GMTThis is an informal research note. It is the result of a few-day exploration into positional SAE features conducted as part of Neel Nanda’s training phase of the ML Alignment & Theory Scholars Program - Summer 2024 cohort.Thanks to Andy Arditi, Arthur Conmy and Stefan Heimersheim for helpful feedback. Thanks to Joseph Bloom for training this SAE.SummaryFigure...

Published on July 28, 2024 5:59 PM GMT"Making Beliefs Pay Rent (in Anticipated Experiences)" is one of the most used lessons from Yudkowsky's Sequences.* First, we're going to go over the essay itself. Second, we're going to suggest beliefs we hold. Third, we're going to be specific about what experiences those believes cause us to anticipate.Suggested reading: Making beliefs pay rent: https://www.lesswrong.com/s/7gRSERQZbqTuLX5re/p/a7n8GdKiAZRX86T5ABelief in Belief:...

Published on July 28, 2024 5:13 PM GMTCross-posted from Substack.I.And the sky opened, and from the celestial firmament descended a cube of ivory the size of a skyscraper, lifted by ten thousand cherubim and seraphim. And the cube slowly landed among the children of men, crushing the frail metal beams of the Golden Gate Bridge under its supernatural weight. On its surface were inscribed...

Published on July 28, 2024 12:23 PM GMTI refer to these posts:https://optimists.ai/2023/11/28/ai-is-easy-to-control/https://www.lesswrong.com/posts/hvz9qjWyv8cLX9JJR/evolution-provides-no-evidence-for-the-sharp-left-turnhttps://www.lesswrong.com/posts/CoZhXrhpQxpy9xw9y/where-i-agree-and-disagree-with-eliezerMy (poor, maybe mis-) understanding is that the argument is that as SGD optimizes for "predicting the next token" and we select for systems with very low loss by modifying every single parameter in the neural network (which basically defines the network itself), it seems quite unlikely that we'll have a "sharp left...

Published on July 28, 2024 7:05 AM GMTThe PDF version can be read here.The audio version can be listened to here:Humans have the capacity to form different types of relationships, which are mediated by emotions. We can divide these relationships into two categories: family and social. They create two types of structure: family and society. The family is held together by the pair bond...