~www_lesswrong_com | Bookmarks (669)

Published on August 6, 2024 2:32 AM GMTThis is a story about a flawed Manifold market, about how easy it is to buy significant objective-sounding publicity for your preferred politics, and about why I've downgraded my respect for all but the largest prediction markets.I've had a Manifold account for a while, but I didn't use it much until I saw and became irked by...

Published on August 6, 2024 1:23 AM GMTSchulman writes:I shared the following note with my OpenAI colleagues today: I've made the difficult decision to leave OpenAI. This choice stems from my desire to deepen my focus on AI alignment, and to start a new chapter of my career where I can return to hands-on technical work. I've decided to pursue this goal at Anthropic, where...

Published on August 5, 2024 10:20 PM GMTTL;DRWe apply the method of SelfIE/Patchscopes to explain SAE features – we give the model a prompt like “What does X mean?”, replace the residual stream on X with the decoder direction times some scale, and have it generate an explanation. We call this self-explanation.The natural alternative is auto-interp, using a larger LLM to spot patterns in max...

Published on August 5, 2024 9:28 PM GMT1. Introduction“Value fragility,” as I’ll construe it, is the claim that slightly-different value systems tend to lead in importantly-different directions when subject to extreme optimization. I think the idea of value fragility haunts the AI risk discourse in various ways – and in particular, that it informs a backdrop prior that adequately aligning a superintelligence requires an...

Published on August 5, 2024 6:36 PM GMTThis year's Fall ACX Meetup in Madrid.Location: El Retiro Park, puppet theatre (https://www.esmadrid.com/en/tourist-information/teatro-de-titeres-de-el-retiro) – https://plus.codes/8CGRC897+F8MContact: pvillalobos@proton.meDiscuss

Published on August 5, 2024 6:10 PM GMT The idea that circular reasoning is bad is widespread. However, this reputation is undeserved. While circular reasoning should not be convincing (at least not usually), it should also not be considered invalid.Circular Reasoning is ValidThe first important thing to note is that circular reasoning is logically valid. A implies A. If circular arguments are to be critiqued,...

Published on August 5, 2024 3:38 PM GMTVitalik Buterin wrote an impactful blog post, My techno-optimism. I found this discussion of one aspect on 80,00 hours much more interesting. The remainder of that interview is nicely covered in the host's EA Forum post.My techno optimism apparently appealed to both sides, e/acc and doomers.  Buterin's approach to bridging that polarization was interesting.  I hadn't understood...

Published on August 5, 2024 1:15 PM GMTAGI is not discrete, and different phases lead to different opportunities, risks, and strategies“How long until AGI? I’d say -1 years from now.”“Considering the worst-case outcomes of AGI, I’m most concerned about the effect on jobs.”“AGI will be like corporations, so we will control it like corporations.”“AGI will be millions of times smarter than us and take...

Published on August 5, 2024 1:00 PM GMTI'm starting a new blog where I post my (subjective) highlights of AI safety paper each month, called "AI Safety at the Frontier". I've been doing this non-publicly for the last year, so I've backfilled highlights and collections up to September 2023.My selection primarily covers ML-oriented research. It's only concerned with papers (arXiv, conferences etc.), not LessWrong...

Published on August 5, 2024 4:27 AM GMTGame theory is a branch of mathematics that deals with decision making when there are multiple decision-makers, called “players”, and outcomes depend on the decisions of other players. Game theory is based on the metaphor of a game, in which the rules of play and the outcomes are well defined. The game metaphor is a very useful...

Published on August 4, 2024 8:47 PM GMTThere is a stark difference between rehearsing classical AI risk 101 arguments about instrumental convergence, and tackling problems like "Design and implement the exact evaluations we'll run on GPT-5 to determine whether it's capable enough that we should worry about it acting aligned until it can execute a takeover". And naturally, since I've started working on problems like...

Published on August 4, 2024 2:39 PM GMTWhile thinking about high-dimensional spaces and their less intuitive properties, I came to the realization that even three spatial dimensions possess the potential to overwhelm our basic human intuitions. This post is an exploration of the gap between actual 3D space, and our human capabilities to fathom it. I come to the conclusion that this gap is actually...

Published on August 4, 2024 2:12 PM GMTTL;DR: Your discernment in a subject often improves as you dedicate time and attention to that subject. The space of possible subjects is huge, so on average your discernment is terrible, relative to what it could be. This is a serious problem if you create a machine that does everyone's job for them.See also: Reality has a...

Published on August 4, 2024 2:05 PM GMTThere is an interesting paragraph in Anthropic's most recent study on persuasion, Measuring the Persuasiveness of Language Models, by Durmus et al. (April 2024):Automated evaluation for persuasiveness is challenging - We attempted to develop automated methods for models to evaluate persuasiveness in a similar manner to our human studies: generating claims, supplementing them with accompanying arguments, and...

Published on August 3, 2024 12:39 PM GMTChapter 2 of Pearl's Causality book claims you can recover causal models given only the observational data, under very natural assumptions of minimality and stability[1].In graphical models lingo, Pearl identifies a causal model of the observational distribution with the distribution's perfect map (if they exist).But I'm confused about a pretty fundamental point: "What does this have to...

Published on August 3, 2024 12:07 PM GMT From the research & engineering team at Leap Laboratories (incl. @Arush, @sebastian-sosa, @Robbie McCorkell), where we use AI interpretability to accelerate scientific discovery from data. This post is about our LLM attribution repo PIZZA: Prompt Input Z? Zonal Attribution. (In the grand scientific tradition we have tortured our acronym nearly to death. For the crimes of others see...

Published on August 3, 2024 10:16 AM GMTThis work was facilitated by the Oxford AI Safety and Governance group, Cooperative AI Foundation, and Oxford Autonomous Intelligent Machines and Systems. Thanks also to Bart Jaworski, Jesse Clifton, Joar Skalse, Sam Barnett, Vincent Conitzer, Charlie Griffin, David Hyland, Michael Wooldridge, Ted Turocy, and Alessandro Abate. This blogpost accompanies the paper Cooperation and Control in Delegation Games...

Published on August 3, 2024 7:20 AM GMTDay OneWe've been handed this old legacy system called "Democracy". It's an emergency. The old maintainers are saying it has been misbehaving lately but they have no idea how to fix it. We've had a meeting with them to find out as much as possible about the system, but it turns out that all the original team...

Published on August 2, 2024 10:35 PM GMTBasically, the user is shown a splatter of colored circles, then the splatter is hidden, and then they're asked to remember what proportion of the splatter was a particular color. To get good at it, they'd have to get good at accurately perceiving and remembering features of entire distributions. Obvious high propensity for transfer to mentally visualizing...

Published on August 2, 2024 7:50 PM GMTThis blog post discusses a collaborative research paper on sparse autoencoders (SAEs), specifically focusing on SAE evaluations and a new training method we call p-annealing. As the first author, I primarily contributed to the evaluation portion of our work. The views expressed here are my own and do not necessarily reflect the perspectives of my co-authors. You...

Published on August 2, 2024 5:53 PM GMTEthical Deception: Should AI Ever Lie?Personal Artificial Intelligence Assistants (PAIAs) are coming to your smartphone. Will they always tell the truth?  Given the future scenario where humans increasingly seek subjective feedback from AIs, we can expect that their influence will accelerate. Will the widespread use of PAIAs influence social norms and expectations around praise, encouragement, emotional support, beauty,...

Published on August 2, 2024 6:39 PM GMTRead the associated paper "Safetywashing: Do AI Safety BenchmarksActually Measure Safety Progress?": https://arxiv.org/abs/2407.21792Focus on safety problems that aren’t solved with scale.Benchmarks are crucial in ML to operationalize the properties we want models to have (knowledge, reasoning, ethics, calibration, truthfulness, etc.). They act as a criterion to judge the quality of models and drive implicit competition between researchers....

Published on August 2, 2024 5:49 PM GMT@KatjaGrace, Josh Hart I are finding quotes around different arguments for AI being an existential riskFull list here: https://docs.google.com/spreadsheets/d/1yB1QIHtA-EMPzqJ_57RvvftvXHTI5ZLAy921Y_8sn3U/edit Currently we are struggling to find proponents of the following arguments:"Loss of control via speed" - that things that might otherwise go well are going to go badly because they are happening so fast"Loss of control via inferiority" -...

Published on August 2, 2024 5:47 PM GMTThis post presents a simple toy coherence theorem, and then uses it to address various common confusions about coherence arguments.SettingDeterministic MDP. That means at each time t.mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width:...