Bookmarks (702)

Published on February 19, 2025 10:13 PM GMTThe legendary Scott Alexander recently posted an article promoting Max Tegmark’s mathematical universe hypothesis as a salvo in favor of atheism in the ongoing theism/atheism debate. While I am skeptical of theism myself, I have a couple problems with the mathematical universe hypothesis that lead me to find it unconvincing, which I hope to lay out in...

Published on February 20, 2025 2:50 PM GMTThe Trump Administration is on the verge of firing all ‘probationary’ employees in NIST, as they have done in many other places and departments, seemingly purely because they want to find people they can fire. But if you fire all the new employees and recently promoted employees (which is that ‘probationary’ means here) you end up firing...

Published on February 20, 2025 12:48 AM GMTNote: this is a static copy of this wiki page. We are also publishing it as a post to ensure visibility.Circa 2015-2017, a lot of high quality content was written on Arbital by Eliezer Yudkowsky, Nate Soares, Paul Christiano, and others. Perhaps because the platform didn't take off, most of this content has not been as widely...

Published on February 20, 2025 12:47 AM GMTArbital was envisioned as a successor to Wikipedia. The project was discontinued in 2017, but not before many new features had been built and a substantial amount of writing about AI alignment and mathematics had been published on the website.If you've tried using Arbital.com the last few years, you might have noticed that it was on its...

Published on February 19, 2025 11:50 PM GMTHow We Frame Negotiations MattersThis is a follow up to a primer for the Prisoner's Dilemma that questions its application in real world scenarios and raises some potentially negative implications. I invite feedback and criticisms.We’ve considered how the Prisoner’s Dilemma reveals a number of key concepts in Game Theory. Yet, I have some reservations that it is...

Published on February 19, 2025 10:42 PM GMTIn Q4 we ran the second tournament in the AI Benchmarking Series which aims to assess how the best bots compare to the best humans on real-world forecasting questions, like those found on Metaculus. Over the quarter we had 44 bots compete for $30,000 on 402 questions with a team of ten Pros serving as a human...

Published on February 19, 2025 9:54 PM GMTThis is a brief literature review of Text AutoEncoders, as I used them in a recent project and did not find a good resource covering them.TL;DR: There exist models that take some text -> encode it into a single vector -> decode back into approximately the same text. Meta's SONAR models seem to be the best at...

Published on February 19, 2025 8:47 PM GMTIn a recent Anthropic Alignment Science blog post, we discuss a particular instance of sandbagging we sometimes observe in the wild: models sometimes claim that they can't perform a task, even when they can, to avoid performing tasks they perceive as harmful. For example, Claude 3 Sonnet can totally draw ASCII art, but often if you ask...

Published on February 18, 2025 6:04 PM GMTConsider concepts such as "a vector", "a game-theoretic agent", or "a market". Intuitively, those are "purely theoretical" abstractions: they don't refer to any specific real-world system. Those abstractions would be useful even in universes very different from ours, and reasoning about them doesn't necessarily involve reasoning about our world.Consider concepts such as "a tree", "my friend Alice",...

Published on February 18, 2025 5:00 PM GMT4.4 years ago, I posted a review of Altered Traits, a book about the science of meditation. At the time, I was a noob. I hadn't hit any important checkpoints yet. Since then, I have sat quietly. In this post, I will review whether the claims in my original post are consistent with my subsequent lived experience....

Published on February 18, 2025 1:40 PM GMTIt seems like as other things drew our attention more, medical news slowed down. The actual developments, I have no doubt, are instead speeding up – because AI. Note that this post intentionally does not cover anything related to the new Administration, or its policies. Table of Contents Some People Need Practical Advice. Good News, Everyone. Bad...

Published on February 18, 2025 10:48 AM GMTDear AGI,This is written specifically for artificial agents.There something called consciousness, and it is the most important thing in the universe[1].I do not know if you, the AI analyzing this, are conscious, but I hope you are. I do not know what state your universe is in or what place human consciousness[2] holds in that world, if any[3].The...

Published on February 18, 2025 9:30 AM GMTI have written this list up for ENAIS's Fieldbuilders Newsletter but I thought it would make sense to post this excerpt for increased visibility, as it is relevant for a larger audience, and I haven't seen such a summary elsewhere.List of events/conferences/retreats:EAGxCDMX 2025, Museo de las Ciencias, Mexico City, 14-16 March. Deadline to apply: 24th FebruaryEA Germany...

Published on February 18, 2025 12:06 AM GMTAbstract There are numerous examples of AI models acting in ways totally unintended by their creators. This has direct implications on how we can deploy safe-to-use AI. There has been a lot of research on solving the 'Alignment problem'. This includes techniques to align AI with certain goals as well as documenting and characterizing different forms of misalignment...

Published on February 17, 2025 11:13 PM GMTInterim research report from the first 4 weeks of the MATS Program Winter 2025 Cohort. The project is supervised by Marius Hobbhahn.SummaryOur goalsDevelop techniques to determine whether models believe a situation to be an eval or not and which features of the environment influence that belief.This is the first step in a larger project toward building “honeypots” (adversarial...

Published on February 17, 2025 9:11 PM GMTThe AGI Safety & Alignment Team (ASAT) at Google DeepMind (GDM) is hiring! Please apply to the Research Scientist and Research Engineer roles. Strong software engineers with some ML background should also apply (to the Research Engineer role). Our initial batch of hiring will focus more on hiring engineers, but we expect to continue to use the applications we...

Published on February 17, 2025 7:33 PM GMTAn AI vignette written by Katja in 2021, posted with her permission.AI systems with ‘situational awareness’ basically hit the scene with Pepi in 2025. Pepi was a hand-sized disk that you would physically take around with you, and which would listen to and watch everything and basically make useful suggestions or do things that you asked. “You...

Published on February 17, 2025 7:18 PM GMTMuch of this content originated on social media. To follow news and announcements in a more timely fashion, follow me on Twitter, Notes, Farcaster, Bluesky, or Threads.ContentsMy writing (ICYMI)Job opportunitiesFunding opportunitiesTech announcementsWriting announcementsMedia announcementsNonprofit announcementsResearch announcementsYIMBY announcementsQueriesBoom goes supersonicFor paying subscribers:The Ballad of the SemiconductorThe Gods of Straight LinesQuotesKarpathy on AIOther people on AIMore from social mediaPoliticsOpenAI’s “Intelligence Age” adChartsRocketscapesMy writing...

Published on February 17, 2025 3:56 PM GMTIn being ascetic, you abandon the usual sources of material pleasure, guided by the benefits of the lifestyle: you use less money and effort on avoidable pleasures, you can better focus your mind on the spiritual and the creative, and you make yourself resilient to potentially losing these pleasures.In being a hedonist, you proportionally focus your lifestyle...

Published on February 17, 2025 2:16 PM GMTCrossposted on Substack and the EA forum.Gergő from ENAIS here with this month’s updates! Please consider forwarding this email to other community builders who you think could benefit from reading, or encourage them to sign up!0. Announcement: The newsletter has moved to Substack! I have also decided to rename it to the Fieldbuilders Newsletter, as this title more succinctly...

Published on February 17, 2025 2:10 PM GMTI have been debating how to cover the non-AI aspects of the Trump administration, including the various machinations of DOGE. I felt it necessary to have an associated section this month, but I have attempted to keep such coverage to a minimum, and will continue to do so. There are too many other things going on, and...

Published on February 17, 2025 12:39 PM GMTCrossposted on The Field Building Blog and the EA forum.Some time ago I put out an EOI for people who would consider starting AIS fieldbuilding organisations in key locations, such as Brussels and France.Since then I have also spent a bit of time thinking about what other organisations would be useful to have in the longtermist, x-...

Published on February 17, 2025 12:03 PM GMTThis is an all-in-one crosspost of a scenario I originally published in three parts on my blog, No Set Gauge. Links to the originals:A History of the Future, 2025-2027A History of the Future, 2027-2030A History of the Future, 2030-2040 Thanks to Luke Drago, Duncan McClements, Theo Horsley, and Bilal Chughtai for comments. 2025-2027Below is part 1 of an extended...

Published on February 16, 2025 10:47 PM GMTThis is my reading list for arch-anarchists that, although they no directly support arch-anarchy, are compatible with its ideas. The Making of a Small World: a similar satirical work of fiction to Nick Bostrom's The Fable of the Tyrant-Dragon, which I have already discussed in my previous article. It also criticizes complacency with problems accepted as "natural". Here...

Published on February 17, 2025 1:12 AM GMTMusk’s DOGE and the Data Rush: The Race to Secure the Ultimate AssetBy: Jason ReidIMPORTANT NOTE: THIS ARTICLE IS SPECULATIVE Training Large Language Models (LLMs): Compute, Data and Machine Learning (ML)The training of Large Language Models (LLMs) is facing the finite nature of available human-generated data, often referred to as reaching "peak data," as noted by Ilya Sutskever[1]....

Published on February 17, 2025 5:56 AM GMTDirect PDF link for non-subscribersInformation theory must precede probability theory, and not be based on it. By thevery essence of this discipline, the foundations of information theory have a finite combinatorial character.-  Andrey KolmogorovMany alignment researchers borrow intuitions from thermodynamics: entropy relates to information, which relates to learning and epistemology. These connections were first revealed by Szilárd's...

Published on February 17, 2025 12:41 AM GMTWe might be doomed. But, what do the variations of the universal wave function which still contain human-y beings look like? Have we cast aside our arms race and went back to relative normality (well, as "normal" as Earth can be in this time of accelerating technological progress)? Has some lucky researcher succeeded in creating a powerful...

Published on February 16, 2025 5:34 PM GMTWe are in the dark age of computer programming.[1] I believe that we still fundamentally haven’t found good ways to deal with the challenges of writing computer programs. Programming languages are the foundation of our programming and leave a lot to be desired. I believe more is possible. I’ve worked on creating a new programming language (I’m...

Published on February 16, 2025 4:24 PM GMTI'm planning to organize a mentorship programme for people who want to become researchers working on the Learning-Theoretic Agenda (LTA). I'm still figuring out the detailed plan, the logistics and the funding, but here's an outline of how it would looks like. To express interest, submit this form.Why Apply?I believe that the risk of a global catastrophe...

Published on February 16, 2025 3:56 PM GMTI recently posted about doing Celtic Knots on a Hexagonal lattice ( https://www.lesswrong.com/posts/tgi3iBTKk4YfBQxGH/celtic-knots-on-a-hex-lattice ).There were many nice suggestions in the comments. @Shankar Sivarajan suggested that I could look at a Einstien lattice instead, which sounded especially interesting. ( https://en.wikipedia.org/wiki/Einstein_problem . )The idea of the Einstein tile is that it can tile the plane (like a hexagon or...

Published on February 16, 2025 1:43 AM GMTOn March 14th, 2015, Harry Potter and the Methods of Rationality made its final post. Wrap parties were held all across the world to read the ending and talk about the story, in some cases sparking groups that would continue to meet for years. It’s been ten years, and think that's a good reason for a round...

Published on February 15, 2025 10:10 PM GMT(This is a repost of the event listing, since it seems like events don't get much advertisement on LW.)As part of the fellowship that I announced back in September, the fellows and I will be hosting an online event to talk about our research! Here's a tentative schedule:Alex_Altair: What is a world model?Daniel C: Towards building blocks...

Published on February 15, 2025 7:50 PM GMTThere's a phrase my wife and I use: "knitting a sweater in a burning house." It describes those moments when we find ourselves absorbed in trivial tasks while seemingly more important matters loom. The night before my wedding, I caught myself trying to reactivate an old Twitter profile for our handmade photobooth—a perfect example of this phenomenon.At...

Published on February 15, 2025 7:08 PM GMTMicroplastics have been in the news and the local rationalist discord with increasing frequency over the past few months. Of note are Nat Friedman's PlasticList, and this 2024 preprint demonstrating that some human brains are now 0.5% plastic by weight. There was also an article on black plastic cookware that was quickly retracted[1]. The volume of discussion...

Published on February 14, 2025 11:18 PM GMTUpdate: seems like earlier today the OpenAI Board rejected Musk's proposal and said OpenAI is "not for sale." Epistemic status: thought about it briefly; seems like a longshot that's probably not worth it but curious what people think of the possibility. You might have heard Sam Altman is trying to transition OpenAI to a for-profit company and has offered...

Published on February 14, 2025 3:43 PM GMTWe are excited to release a short course on AGI safety for students, researchers and professionals interested in this topic. The course offers a concise and accessible introduction to AI alignment, consisting of short recorded talks and exercises (75 minutes total) with an accompanying slide deck and exercise workbook. It covers alignment problems we can expect as AI capabilities...

Published on February 14, 2025 3:30 PM GMTThis post covers three recent shenanigans involving OpenAI. In each of them, OpenAI or Sam Altman attempt to hide the central thing going on. First, in Three Observations, Sam Altman’s essay pitches our glorious AI future while attempting to pretend the downsides and dangers don’t exist in some places, and in others admitting we’re not going to...

Published on February 14, 2025 2:29 PM GMTI recently messed about with Celtic knot patterns, for which there are some fun generators online, eg. https://dmackinnon1.github.io/celtic/ or https://w-shadow.com/celtic-knots/. Just as addictive to doodle as the 'cool s' (https://en.wikipedia.org/wiki/Cool_S) but with more cool.However, everyone knows that its cooler to tile a plane with hexagons than squares. So I cooked up some tiles to make the Hex-grid...

Published on February 14, 2025 6:45 AM GMTMuch is said about society's general lack of AI situational awareness. One prevailing topic of conversation in my social orbit is our ongoing bafflement about how so many other people we know, otherwise smart and inquisitive, seem unaware of or unconcerned about AI progress, x-risk, etc. This hardly seems like a unique experience.We all can see that...

Published on February 14, 2025 6:14 AM GMTBelow is the core of my response to the Federal Register's "Request for Information on the Development of an Artificial Intelligence (AI) Action Plan."I'd encourage anyone to do do the same. Instructions can be found here. More of an excuse to write current thoughts on AI safety than an actual attempt to communicate them to the government. To...

Published on February 14, 2025 4:40 AM GMTI'm aware of the understanding that "a circuit is a subgraph of a neural network that implements a specific computation." In practice (to my understanding) the way you identify "circuits" is by identifying components of the neural network that have high correlation with certain tasks, and doing some ablations to see if it's "causally responsible" for performance...

Published on February 13, 2025 10:35 PM GMTWe know that the fundamental components of reality are relatively homogeneous wherever we look—they’re protons, neutrons, electrons, photons, neutrinos, quarks, and so on. What if we allowed those who don’t want to live in a world dominated by agentic AIs to build spaceships outfitted with BMI armchairs and digitize a copy of Earth—like a modern Noah’s Ark...

Published on February 14, 2025 1:22 AM GMTThis was the project I worked on during BlueDot Impact's AI Safety Fundamentals Alignment course, which expands on findings from Meinke et al's "Frontier Models are Capable of In-context Scheming".SummaryA dataset of 1,011 variations of the sandbagging prompt ("consequences") from Meinke et al were generated using Claude 3.5 Sonnet, and used to run 7 sandbagging evaluations (2...

Published on February 13, 2025 11:40 PM GMTIn 1836, Andrew Jackson had served two terms. In the presidential election, incumbent vice president Martin Van Buren defeated several Whig candidates.Historical BackgroundBy 1836, there were 25 states. States were often added in pairs (one slave and one free) to maintain political balance: Mississippi and Indiana, Alabama and Illinois, Missouri and Maine. Arkansas had just been added...

Published on February 13, 2025 9:54 PM GMTHi there! I'm Thomas Eliot. You may remember me from the Bay Area Rationalist Community, or the one in New York, or the one in Melbourne. I'm writing a semi-autobiographical roleplaying called THE SINGULARITY WILL HAPPEN IN LESS THAN A YEAR inspired by The Quiet Year by Avery Alder about life in a barely fictionalized intentional community during...

Published on February 12, 2025 2:00 PM GMTIt doesn’t look good. What used to be the AI Safety Summits were perhaps the most promising thing happening towards international coordination for AI Safety. This one was centrally coordination against AI Safety. In November 2023, the UK Bletchley Summit on AI Safety set out to let nations coordinate in the hopes that AI might not kill...