Bookmarks (663)

Published on November 20, 2024 6:19 PM GMTI am grateful to Noosphere89 for prompting me to start this discussion. In human evolution, the fundamental goal was to survive and reproduce, passing on our genes to the next generation. But somewhere along the way, humans developed complex societies and technologies that go beyond just survival and reproduction. Now, we often find ourselves pursuing goals that...

Published on November 20, 2024 4:48 PM GMTCrosspost of this on my blog. 1 My basic view 'Cause everybody criesEverybody hurts sometimesSometimes everything is wrong—Rem “Everybody hurts”You didn’t think the era of absurdly long posts was over, did you?Prior to the 2024 election, there were a series of swing states that pundits forecasted could have gone either way. All of them ended up going to Trump...

Published on November 20, 2024 1:40 PM GMTPreviously: Long-Term Charities: Apply For SFF Funding, Zvi’s Thoughts on SFF There are lots of great charitable giving opportunities out there right now. I recently had the opportunity to be a recommender in the Survival and Flourishing Fund for the second time. As a recommender, you evaluate the charities that apply and decide how worthwhile you think...

Published on November 20, 2024 11:48 AM GMTAuthors: Anonymous (I'm not one of them).Abstract:Most analysis of transformer expressivity treats the depth (number of layers) of a model as a fixed constant, and analyzes the kinds of problems such models can solve across inputs of unbounded length. In practice, however, the context length of a trained transformer model is bounded. Thus, a more pragmatic question...

Published on November 20, 2024 11:04 AM GMTCurrently, it looks like Robert F. Kennedy Jr. will become HHS secretary and change a lot of things at the HHS. There will likely be both good and bad changes.It's a time when the ideas that lay around matter. What changes do you think should be made? What clever policy ideas should be adopted?Discuss

Published on November 20, 2024 10:13 AM GMTSummary thread: https://x.com/GarrisonLovely/status/1859022323799699474. An influential congressional commission is calling for a militarized race to build superintelligent AI based on threadbare evidenceDiscuss

Published on November 20, 2024 6:04 AM GMTI run a weekly sequences-reading meetup with some friends, and I want to add a film-component, where we watch films that have some tie-in to what we've read.I got to talking with friends about what good rationality films there are. We had some ideas but I wanted to turn it to LessWrong to find out.So please, submit...

Published on November 20, 2024 1:37 AM GMTAs I relayed in my last post, in "Theory of Games and Economic Behavior", von Neumann and Morgenstern defended the validity of using utility to predict economic outcomes despite the fact that cross-individual utility cannot apparently be measured or apprehended directly: "The [ . . . ] difficulties of the notion of utility, and particularly of the...

Published on November 20, 2024 12:57 AM GMTI work in a niche company in a niche industry mostly with a nearly-obsolete programming language. I have worked for the same company for an extremely long time by programmer standards, basically right out of high-school. I have not looked for work in a very long time.Reading HackerNews lately, the market looks pretty grim. I am reading...

Published on November 18, 2024 6:25 PM GMTWe wanted to run an event where controversial opinions could be shared without restraint: “a safe space for what you can't say in ‘safe spaces’”So a mechanism was devised:https://x.com/tobi/status/1795619548579918222Examples of what may or may have been on the slips:“Only white men can save us.”“Intelligent people are worth more.”“Bestiality is good, actually.” We also advertised the event so only...

Published on November 18, 2024 4:00 PM GMTWhat kinds of ethical implications should we expect from the Many-Worlds Interpretation of Quantum Mechanics (MWI)? I’ll argue that we shouldn’t expect decision-making to change. The implications are more about how we should think or feel about events in our lives, and the virtues of taking a cosmic perspective.According the many-worlds interpretation (MWI) of quantum mechanics, the universe...

Published on November 18, 2024 2:25 PM GMTGreat. Another crucial consideration I missed. I was convinced that working on reducing the existential risk for humanity should be a global priority.Upholding our potential and ensuring that we can create a truly just future seems so wonderful.Well, recently I was introduced to the idea that this might actually not be the case. The argument is rooted in...

Published on November 18, 2024 4:58 PM GMTPeople often ask me this. It’s a good question. It seems potentially important for anyone who doesn’t want to die, or doesn’t want other people to have to die. I don’t feel like I have a good answer, which is one reason I want to do more research. But recently I think my answer has gotten a...

Published on November 18, 2024 4:05 PM GMT(thanks to Alex Mallen, Cody Rushing, Zach Stein-Perlman, Hoagy Cunningham, Vlad Mikulik, and Fabien Roger for comments)Sometimes I hear people argue against AI control as follows: if your control measures rely on getting good judgments from "trusted" AI models, you're doomed because AI models aren't adversarially robust. I agree that better adversarial robustness would definitely help with...

Published on November 18, 2024 1:20 PM GMTThis is your monthly roundup. Let’s get right to it. Young People are Young and Stupid As a reminder that yes college students are often young and stupid and wrong about everything, remember the time they were behind a ban on paid public toilets? This is a central case of the kind of logic that often gets...

Published on November 18, 2024 12:45 PM GMTThis post was written during the agent foundations fellowship with Alex Altair funded by the LTFF. Thanks to Alex, Jose, Daniel, Cole, and Einar for reading and commenting on a draft.The Good Regulator Theorem, as published by Conant and Ashby in their 1970 paper (cited over 1700 times!) claims to show that 'every good regulator of a...

Published on November 18, 2024 12:30 PM GMTOn the emergence of history's reinsOne general law, leading to the advancement of all organic beings, namely, multiply, vary, let the strongest live and the weakest die.― Charles DarwinWe are a way for the universe to know itself.— Carl SaganIf you don’t know where you are going, you’ll end up someplace else.— Yogi BerraImagine you were trying...

Published on November 18, 2024 4:21 AM GMTFertility rates in the developed world are too low and that's bad, blah blah.Here's my proposal to raise fertility rates (coming from a current uni student):Promote parenthood in universities via student run clubs. Reading groups about different parenting styles, or more subtle pro-fertility clubs like knitting groups where students make clothes for sick kids (very un-EA, I...

Published on November 18, 2024 2:13 AM GMTNote: I would have done this as a Quick Take, but it doesn't allow images.In the spirit of How I got 4.2M YouTube views without making a single video, I improved a diagram on the wikipedia page for Pareto Efficiency.ORIGINAL VERSIONNEW VERSION Based on the text, I don't think the author understood that Cooperate/Cooperate isn't the only Pareto...

Published on November 16, 2024 2:31 PM GMTAt every turn there was a Gwern, Within this dream I couldn't discern If to Heaven I did rise Or in Satan's pit - my soul's demise. Creatures short and stout and keen of mind Wrote and wrote, words flowed like wine. "Good sir," I said, "how did you come to be In this place with him....

Published on November 16, 2024 2:24 PM GMTI want to make a serious effort to create a bigger evals field. I’m very interested in which resources you think would be most helpful. I’m also looking for others to contribute and potentially some funding. Which resources would be most helpful? Suggestions include:Open Problems in Evals list: A long list of open relevant problems & projects in...

Published on November 16, 2024 6:38 AM GMTAs part of the court case between Elon Musk and Sam Altman, a substantial number of emails between Elon, Sam Altman, Ilya Sutskever, and Greg Brockman have been released. I have found reading through these really valuable, and I haven't found an online source that compiles all of them in an easy to read format. So I made...

Published on November 16, 2024 4:29 AM GMTRob Miles has released a new video, this time on control!Youtube Transcript (Very rough)0:00hi so this channel is about AI safety a lot of people when they hear AI safety0:05the first thing that comes to mind is they think it's about how to keep potentially dangerous AI systems under control that we're trying to answer0:11questions like what...

Published on November 16, 2024 2:59 AM GMTEpistemic status: Toy model. Oversimplified, but has been anecdotally useful to at least a couple people, and I like it as a metaphor.IntroductionI’d like to share a toy model of willpower: your psyche’s conscious verbal planner “earns” willpower (earns trust with the rest of your psyche) by choosing actions that nourish your fundamental, bottom-up processes in the...

Published on November 16, 2024 12:57 AM GMTI wrote a whole book! What's next?I'm currently doing an edit pass on the entire book. I need to rewrite some of the early sections, fix some consistency issues, and generally look with fresh eyes on words I wrote months or years ago. Many of you provided helpful comments, and I'm using those to make the second...

Published on November 15, 2024 6:55 PM GMTTrump and the Republican party will yield broad governmental control during what will almost certainly be a critical period for AGI development. In this post, we want to briefly share various frames and ideas we’ve been thinking through and actively pitching to Republican lawmakers over the past months in preparation for the possibility of a Trump win.Why...

Published on November 15, 2024 3:47 PM GMTIn this post, I’ll make a technical point that comes up when thinking about risks from scheming AIs from a control perspective.In brief: Consider a deployment of an AI in a setting where it’s going to be given a sequence of tasks, and you’re worried about a safety failure that can happen suddenly. Every time the AI...

Published on November 15, 2024 1:59 PM GMTTechnological progress can excite us, politics can infuriate us, and wars can mobilize us. But faced with the risk of human extinction that the rise of artificial intelligence is causing, we have remained surprisingly passive. In part, perhaps this was because there did not seem to be a solution. This is an idea I would like to...

Published on November 14, 2024 6:39 PM GMTI spent most of my election day -- 3pm to 11pm Pacific time -- trading on Manifold Markets. That went about as well as it could have gone. I doubled the money I was trading with, jumping to 10th place on Manifold's all-time leaderboard. Spending my time trading instead of just nervously watching results come in also...

Published on November 14, 2024 4:59 PM GMTAbstractThis post summarises my findings on the effects of Non-Uniform feature sparsity on Superposition in the ReLU output model, introduced in the Toy Models of Superposition paper, the ReLU output model is a toy model which is shown to exhibit features in superposition instead of a dedicated dimension ('individual neuron') devoted to a single feature. That experiment...

Published on November 14, 2024 5:02 PM GMTScott Alexander's latest today discusses Robert Axelrod's Prisoner’s Dilemma Tournament and the morality of Christians, Quakers, and the rationalist community. I think is of general interest to LessWrong readers.This is why I’m so fascinated by the early Christians. They played the doomed COOPERATE-BOT strategy and took over the world.https://www.astralcodexten.com/p/the-early-christian-strategy Discuss

Published on November 14, 2024 2:37 PM GMT1. PROBLEM In today’s digital era, teenagers face a dual challenge: processing vast amounts of information and maintaining emotional well-being. Traditional educational models, focused on memorization and mechanical procedures, might be better suited to preparing students for complex decision-making in an increasingly uncertain world.As part of a non-profit Latín American education initiative, Jacominesp, I seek to connect with...

Published on November 14, 2024 2:10 PM GMTAs the Trump transition continues and we try to steer and anticipate its decisions on AI as best we can, there was continued discussion about one of the AI debate’s favorite questions: Are we making huge progress real soon now, or is deep learning hitting a wall? My best guess is it is kind of both, that...

Published on November 14, 2024 1:06 PM GMTTLDR:Fluent dreaming for language models is an algorithm based on the GCG method that can reliably find plain-text readable prompts for LLMs that maximize certain logits or residual stream directions by using gradients and genetic algorithms. Authors showed its use for visualizing MLP neurons. We show this method can also help interpret SAE features.We reimplement the algorithm in...

Published on November 14, 2024 7:00 AM GMTYouTube link Do language models understand the causal structure of the world, or do they merely note correlations? And what happens when you build a big AI society out of them? In this brief episode, recorded at the Bay Area Alignment Workshop, I chat with Zhijing Jin about her research on these questions. Topics we discuss: How...

Published on November 14, 2024 6:13 AM GMTFrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AIFrontierMath presents hundreds of unpublished, expert-level mathematics problems that specialists spend days solving. It offers an ongoing measure of AI complex mathematical reasoning progress.We’re introducing FrontierMath, a benchmark of hundreds of original, expert-crafted mathematics problems designed to evaluate advanced reasoning capabilities in AI systems. These problems span major...

Published on November 14, 2024 5:07 AM GMTThanks to Erik Jenner for helpful comments and discussion(Epistemic Status: Tentative take on how to think about heuristic estimation and surprise accounting in the context of activation modeling and causal scrubbing. Should not be taken as authoritative/accurate representation of how ARC thinks about heuristic estimation)I'm pretty excited about ARC's heuristic arguments agenda. The general idea that "formal(ish)...

Published on November 14, 2024 5:01 AM GMTReligions are collections of cherished but mistaken principles. So anything that can be described either literally or metaphorically as a religion will have valuable unexplored ideas in its shadow.-Paul GrahamThis post isn't intended to construct full arguments for any of my "heresies" - I am hoping that you may not have considered them at all yet, but...

Published on November 12, 2024 6:23 PM GMTEpistemic status: Sudden public attitude shift seems quite possible, but I haven't seen it much in discussion, so I thought I'd float the idea again. This is somewhat dashed off since the goal is just to toss out a few possibilities and questions.In Current AIs Provide Nearly No Data Relevant to AGI Alignment, Thane Ruthenis argues that...

Published on November 12, 2024 5:55 PM GMTEpistemic Status: This is a collection of useful heuristics I’ve gathered from a wide range of books and workshops, all rather evidence-based (robustness varies). These techniques are designed to supplement basics of rationalist discourse, helping facilitate interactions—mostly with those unfamiliar with rationalist thought, especially on entry-level arguments. They may also be useful in conversations between rationalists on...

Published on November 12, 2024 5:50 PM GMTOn December 14th, New York City will have a Secular Solstice. Solstice is a holiday for people comfortable with uncomfortable truths and who believe in good. Secular Solstices take place in many cities around the world, but for us New York City is the best place for it. The first Solstice started here, amid towers that reach...

Published on November 12, 2024 5:46 PM GMTSecular Solstice is a celebration of hope in darkness. For more than a decade now, people have gathered in New York City to sing about humanity's distant past and the future we hope to build. You are, of course, invited. This year, Solstice and the traditional Rationalist Megameetup will both be at the Sheraton Brooklyn New York Hotel,...

Published on November 12, 2024 10:56 AM GMTWe’re excited to announce that applications are now open for our 2025 Q1 Pivotal Research Fellowship, a 9-week program designed to enable promising researchers to produce impactful research and accelerate their careers in technical AI safety, AI governance, and biosecurity.About the FellowshipThe Pivotal Research Fellowship is hosted in London at the London Initiative for Safe AI (LISA). It offers...

Published on November 12, 2024 6:45 AM GMT[ This is supposed to be a didactic post. I'm not under the impression that I'm saying anything genuinely new. Thanks to Stephen Wolfram. ] I'm about an hour in to the Yudkowsky-Wolfram discussion [AI-generated transcript from which I'm quoting]. Wolfram thinks we should not fear AI doom very much in particular. I think he is wrong....

Published on November 12, 2024 6:12 AM GMTQuick check: do you agree or disagree with the following statement:If a study finds a result significant at a p=0.05 level, that means they have followed a methodology which produces this conclusion correctly 95 % of the time.Yes or no? Keep that in mind, and we’ll get back to it.I’m reading the Fisher book where he popularised...

Published on November 12, 2024 3:07 AM GMTI.As I've run and studied meetups, there's a useful metaphor that's become more important to how I think about them. For most meetups, there's the packaging, and a payload, and these are related but useful to approach separately. Allow me to expand.The payload is the thing you actually want. If you order some socks off Amazon, the...

Published on November 12, 2024 2:00 AM GMTPeople say "I think" a lot. Here are some examples:I think you brought me the wrong order.I think the numbers in the report are wrong.I think you need to turn left at the light.I think we need to replace the whole water heater.I think iPhones are better than Android phones.I think you should quit your job and...

Published on November 12, 2024 2:00 AM GMT Each year ( 2014, 2015, 2016, 2017, 2018, 2019, 2023) I put out a list of how many dance weekends, festivals, camps, and long dances contra bands and callers are doing. I don't really know why I do this, but it's about an hours work on top of that I'm already collecting for trycontra.com/events so I...