Bookmarks (704)

Published on April 25, 2025 4:02 PM GMTIntroductionEffective Altruism (EA) is a social movement that aims to use reason and evidence to help others as much as possible. It encourages people to ask not just “how to do good”, but how to do the most good. This has led members to support things like global health interventions, existential risk reduction, and animal welfare.I used...

Published on April 25, 2025 3:32 PM GMTSince the 1970s, productivity has outpaced wage growth in many sectors. One sector where wages have kept up with productivity growth is tech, especially among programmers. But, I suspect this is changing thanks to the use of AI coding assistants.To quickly review the relevant economic history, before 1973 wage growth roughly tracked productivity in Western economies, but...

Published on April 25, 2025 2:40 PM GMT At work we've recently been using zstd as a better-compressing alternative to gzip, and overall I've been pretty happy with it. A minor documentation gripe, though, is that the behavior around multithreaded compression is a bit unclear. I understand it's chunking the work and sending chunks to different threads to parallelize the compression process, and this...

Published on April 25, 2025 10:03 AM GMTLetters to attorney generals, etc, to block OpenAI from moving assets into a corporation.Sep: Public CitizenDec: MetaJan, Apr: Californian civil society nonprofitsApr: Coalition for AI Nonprofit IntegrityApr: Safety ex-employees' brief for Musk lawsuitApr: Safety+ researchers on 'not for private gain' Discuss

Published on April 25, 2025 5:12 AM GMTI was recently encouraged to read Jeffrey Heninger's report "Why Did Environmentalism Become Partisan?"  It was interesting, but I thought it had some critical flaws.  I would've recommended rejecting it if I were reviewing it for an academic conference.  I've written a mock review below.  As typical when reviewing for a conference, I didn't aim to mince...

Published on April 24, 2025 9:22 PM GMTTLDR: I really like the graph where they show the best model for every price point. And how Google's models have the best performance at any price point. So I made it live and fetches news data and model regularly because things are moving fast and images don't refresh themselves. (Thanks Claude) https://winston-bosan.github.io/llm-pareto-frontier/(Will be hosted soon in a...

Published on April 24, 2025 9:15 PM GMTIn this post, we study whether we can modify an LLM’s beliefs and investigate whether doing so could decrease risk from advanced AI systems.We describe a pipeline for modifying LLM beliefs via synthetic document finetuning and introduce a suite of evaluations that suggest our pipeline succeeds in inserting all but the most implausible beliefs. We also demonstrate...

Published on April 24, 2025 9:15 PM GMTYesterday, I couldn't wrap my head around some programming concepts in Python, so I turned to ChatGPT (gpt-4o) for help. This evolved into a very long conversation (the longest I've ever had with it by far), at the end of which I pasted around 600 lines of code from Github and asked it to explain them to...

Published on April 24, 2025 8:17 PM GMTWhat in retrospect seem like serious moral crimes were often widely accepted while they were happening. This means that moral progress can require intellectual progress.[1] Intellectual progress often requires questioning received ideas, but questioning moral norms is sometimes taboo. For example, in ancient Greece it would have been taboo to say that women should have the same political...

Published on April 24, 2025 7:09 PM GMTIn late 2019 I wrote a TTRPG.The theme was alien sentience (or perhaps sapience is the technical term), both "organic" (extra terrestrial) and "artificial" (AI/robots). This is the prologue that kicks off play. I found it really fascinating to look back on late 2019 and how I was thinking about some of these topics. Earthdate March 2022. Two months ago...

Published on April 23, 2025 4:05 PM GMTtl;dr: Even if we can't solve alignment, we can solve the problem of catching and fixing misalignment.If a child is bowling for the first time, and they just aim at the pins and throw, they’re almost certain to miss. Their ball will fall into one of the gutters. But if there were beginners’ bumpers in place blocking...

Published on April 23, 2025 12:01 PM GMTDear policymakers,We demand that the AI alignment budget be Belief-Consistent with the military budget.Belief-Consistency is a simple yet powerful idea:If you spend 8000 times less on AI alignment (compared to the military),You must also believe that AI risk is 8000 times less (than military risk).[1]Yet the only way to reach Belief-Consistency, is toGreatly increase AI alignment spending,orBecome...

Published on April 23, 2025 11:06 AM GMTto follow up my philantropic pledge from 2020, i've updated my philanthropy page with the 2024 results.in 2024 my donations funded $51M worth of endpoint grants (plus $2.0M in admin overhead and philanthropic software development). this comfortably exceeded my 2024 commitment of $42M (20k times $2100.00 — the minimum price of ETH in 2024).this also concludes my...

Published on April 23, 2025 3:35 AM GMTWhat would it take to convince you to come and see a fish that recognizes faces?Note: I'm not a marine biologist, nor have I kept fish since I was four. I have no idea what fish can really do. For the purposes of this post, let's suppose that fish recognizing faces is not theoretically impossible, but beyond...

Published on April 23, 2025 5:11 AM GMTI would like to revisit some of the concepts Scott explored in his 2020 article "For, Then Against, High-Saturated-Fat Diets". I'm hoping someone will have some novel/updated insights or new research to share concerning the impacts of the Western diet on health in 2025. I'm about to turn 32, and I sense that I'm moving toward becoming a...

Published on April 23, 2025 4:51 AM GMTGuillaume Blanc has a piece in Works in Progress (I assume based on his paper) about how France’s fertility declined earlier than in other European countries, and how its power waned as its relative population declined starting in the 18th century. In 1700, France had 20% of Europe’s population (4% of the whole world population). Kissinger writes...

Published on April 22, 2025 11:47 PM GMTEpistemic status: Like all alignment ideas, this one is incomplete and/or wrong, but I am hoping mostly incomplete and not wrong.One of the hard subproblems of alignment is constructing a "stable pointer to value" (see this overview and this (sub)sequence discussing OU agents). The way that I think of this is that we want to be able...

Published on April 22, 2025 8:39 PM GMTThe European AI Office is currently writing the rules for how general-purpose AI (GPAI) models will be governed under the EU AI Act. The are explicitly asking for feedback on how to interpret and operationalize key obligations under the AI Act. This includes the thresholds for systemic risk, the definition of GPAI, how to estimate training compute, and when...

Published on April 22, 2025 7:25 PM GMTSummary:Traditional LLMs outperform reasoning models in cooperative Public Goods tasks. Models like Llama-3.3-70B maintain ~90% contribution rates in public goods games, while reasoning-focused models (o1, o3 series) average only ~40%.We observe an "increased tendency to escape regulations" in reasoning models. As models improve in analytical capabilities, they show decreased cooperative behavior in multi-agent settings.Reasoning models more readily opt for...

Published on April 22, 2025 7:04 PM GMTI recently had the privilege of having my idea criticized at the London Institute for Safe AI, including by Philip Kreer and Nicky Case. Previously the idea was vague; being with them forced me to make the idea specific. I managed to make it so specific that they found a problem with it! That's progress :)Reminder: diagrams...

Published on April 22, 2025 6:28 PM GMTThere will be no single moment, no dramatic cinematic climax where humanity loses control. Forget the Hollywood singularity, the sharp left turn into dystopia often breathlessly debated by the very people enabling a slower, more mundane version of it. That’s not how it happens. It’s subtler. More insidious. More… us.You will give it up. Every day. Piece...

Published on April 22, 2025 5:36 PM GMTEach year, Manifund partners with regrantors: experts in the field of AI safety, each given an independent budget of $100k+. Regrantors can initiate fast, small grants, seeding early-stage projects with $5k-$50k.For 2025, we’ve raised $2.25m so far, and are excited to announce our first 10 regrantors:Neel Nanda — DeepMindLisa Thiergart — SL5 Task ForceLauren Mangla — ConstellationAidan...

Published on April 22, 2025 5:08 PM GMTWelcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required.In this edition: AI now outperforms human experts in specialized virology knowledge in a new benchmark; A new report explores the risk of AI-enabled coups.Listen to the AI Safety Newsletter for free on Spotify...

Published on April 22, 2025 2:09 PM GMTCrossposted from my blog In this fictional dialogue between a Bayesian (B) and a Non-Bayesian (N) I will propose solutions to some pre-existing problems with Bayesian epistemology, as well as introduce a new problem for which I offer a solution at the end. (Computer scientists may consider skipping to that section).Here’s a Bayes theorem cheat sheet if you...

Published on April 22, 2025 1:21 PM GMTJoel Z. Leibo [1], Alexander Sasha Vezhnevets [1], William A. Cunningham [1, 2], Sébastien Krier [1], Manfred Diaz [3], Simon Osindero [1][1] Google DeepMind, [2] University of Toronto, [3] Mila Québec AI Institute Disclaimer: These are our own opinions; they do not represent the views of Google DeepMind as a whole or its broader community of safety researchers.Beyond Alignment: The Patchwork Quilt of...

Published on April 22, 2025 1:10 PM GMTOr you had better not. The question is which one. This post covers the announcement of Mechanize, the skeptical response from those worried AI might kill everyone, and the associated (to me highly frustrating at times) Dwarkesh Patel podcast with founders Tamay Besiroglu and Ege Erdil. Mechanize plans to help advance the automation of AI labor, which...

Published on April 22, 2025 12:34 PM GMTTL;DR: Several experiments show that I can extract useful information just by treating myself as a random sample, and thus a view that I can't use myself as a random sample is false. But it's still not clear whether this can be used to prove the Doomsday argument.There are two views: one view is that I can...

Published on April 22, 2025 7:16 AM GMTO3 and Claude 3.7 are terribly dishonest creatures. Gemini 2.5 can be a bit dishonest too, even though Google writes tests like mad. I would bet that lie-proof benchmarks will be difficult and expensive to make and that the lie-proofing techniques won't easily generalize outside of coding tasks. Perhaps a more punishing optimizer would help solve this...

Published on April 22, 2025 5:00 AM GMTThis is a cross-post from https://250bpm.substack.com/p/accountability-sinksBack in the 1990s, ground squirrels were briefly fashionable pets, but their popularity came to an abrupt end after an incident at Schiphol Airport on the outskirts of Amsterdam. In April 1999, a cargo of 440 of the rodents arrived on a KLM flight from Beijing, without the necessary import papers. Because...

Published on April 22, 2025 3:22 AM GMTThis is a linkpost to an article by Ege Erdil and I that we wrote to explain an important perspective that we share regarding AI automation. I'll quote the introduction:A popular view about the future impact of AI on the economy is that it will be primarily mediated through AI automation of R&D. In some form or...

Published on April 21, 2025 3:30 PM GMTThis seemed like a good next topic to spin off from monthlies and make into its own occasional series. There’s certainly a lot to discuss regarding crime. What I don’t include here, the same way I excluded it from the monthly, are the various crimes and other related activities that may or may not be taking place...

Published on April 21, 2025 3:21 PM GMTIn our earlier post, we described how one could parametrize local image patches in natural images by a surface called a Klein bottle.  In Love et al, we used this information to modify the convolutional neural network construction so as to incorporate information about the pixels in a small neighborhood of a given pixel in a systematic...

Published on April 21, 2025 2:10 PM GMTBy Xander BalwitWith death all but obsolete, Jamie’s life felt moot and emaciated. The Obituary Desk at The Times, where he worked, had turned into a ghost town he presided over with the bearing of a man who had given everything up for the bitter disappointment of a mine devoid of mineral riches. He would go on...

Published on April 21, 2025 3:18 AM GMTEpistemic status: exploratory but confident. This essay presents a general framework for identifying and analyzing ideologies based on recurring structural patterns. It draws on observation, theory, and examples from across the political spectrum. The goal is to improve mental defenses against manipulative belief systems, not to discredit all ideological thinking. I welcome critique and refinement.We live in...

Published on April 21, 2025 3:52 AM GMTDisclaimer: this post was not written by me, but by a friend who wishes to remain anonymous. I did some editing, however.So a recent post my friend wrote has made the point quite clearly (I hope) that LLM performance on the simple task of playing and winning a game of Pokémon Red is highly dependent on the...

Published on April 21, 2025 3:09 AM GMTAI 2027 lies at a Pareto frontier – it contains the best researched argument for short timelines, or the shortest timeline backed by thorough research[1]. My own timelines are substantially longer, and there are credible researchers whose timelines are longer still. For this reason, I thought it would be interesting to explore the key load-bearing arguments AI...

Published on April 21, 2025 2:21 AM GMT***Severance Spoilers!***Nick Bostrom talks about coherent, extrapolated ethics as the goal of AI alignment, specifically to avoid calcification from our current moral code, which likely contains many things future generations would find unethical, just as we would previous generations. Since reading that, I've been wondering what things we accept today might alter the trajectory of the future.Another...

Published on April 20, 2025 7:00 PM GMTSay hi at ICSE in Ottawa, I’ll be at the reception Thursday, this colocated event on Friday, and the LLM4Code workshop on Saturday.As usual there are no benefits to the paid subscription. Sorry for consolidating two months into one post again after I said I wouldn’t. Subscribe nowFermstimate of the cost of patching all security relevant open...

Published on April 20, 2025 6:50 PM GMTThe current University System is bad. Very bad. Half of young adults spend 3 to 4 years learning nothing of particular value so that they can get a piece of paper that gives them a leg up when applying for a job. They can't stop, because then somebody else will be first in line for an interview,...

Published on April 19, 2025 6:37 PM GMTThis post summarizes some of the research I have been doing for Bootstrap Bio AKA kman and Genesmith TL;DR We currently don’t have a list of all imprinted genes that are important in human development, but further long-read sequencing in adults and in the human placenta is going to get us close to that goal. Measuring DNA...

Published on April 19, 2025 12:47 AM GMTWe don’t always know what we’re looking for until we stop looking for what we are told to want.When I worked as a relationship coach, most people came to me with a list. A neat, itemised checklist of traits their future partner must have. Tall. Intelligent. Ambitious. Spiritual. Funny but not flippant. Driven but not workaholic. Family-oriented...

Published on April 19, 2025 3:37 PM GMTIn the face of any hard problem—reversing climate change, curing cancer, or starting a great novel—modern LLMs can generate thousands of possible solutions relatively cheaply.Most solutions from most prompts are bad: they’re not new relative to the state of the art, not feasible, or not significant enough.But for every thousand ideas an LLM has about how to...

Published on April 19, 2025 2:47 PM GMTThough, given my doomerism, I think the natsec framing of the AGI race is likely wrongheaded, let me accept the Dario/Leopold/Altman frame that AGI will be aligned with the national interest of a great power. These people seem to take as an axiom that a USG AGI will be better in some way than a CCP AGI....

Published on April 19, 2025 2:09 PM GMTI aim to cover a lot of ground, but only briefly. Click through the links for more detailed explanations.Several of the techniques here see general use outside of interpretability. My definitions/references lean towards how they specifically apply to Mechanistic Interpretability Sparse Dictionary Learning describes any ML technique that aims to find a set of “atoms” that can be...

Published on April 19, 2025 11:40 AM GMT Cross-posted from my NAO Notebook. This is an internal strategy note I wrote in November 2024 that I'm making public with some light editing. In my work at the NAO I've been thinking about what I expect to see as LLMs continue to become more capable and get closer to where they can significantly accelerate their...

Published on April 19, 2025 9:12 AM GMTThe ambulance screeched to a halt with the flair of a rock opera outside an apartment block whose hallway smelled like dry leaves, soggy cardboard, and… was that melted cheese? Best not to investigate. Within seconds, my partner—a guy who had tragedy for breakfast with his coffee—confirmed the inevitable: the mother, lifeless, sprawled in the kitchen next...

Published on April 19, 2025 4:29 AM GMTThis is a living document. Crosspost below may not be updated. Click link for the latest version. 2025-04-19 tldr SecureDrop provides less source privacy than journalist privacy, even though it is the source not the journalist who is more likely to be punished for their actions. SecureDrop opsec guide may be overkill for the median whistleblower but...

Published on April 18, 2025 9:20 PM GMTOpenAI has finally introduced us to the full o3 along with o4-mini. Greg Brockman (OpenAI): Just released o3 and o4-mini! These models feel incredibly smart. We’ve heard from top scientists that they produce useful novel ideas. Excited to see their positive impact on people’s daily lives and humanity’s hardest problems! Sam Altman: we expect to release o3-pro...

Published on April 18, 2025 9:15 PM GMTAI Control is a subfield of AI Safety research focused on ensuring the safety of powerful AI systems, particularly under the assumption that they might be misaligned or could become intentionally subversive (A Brief Explanation of AI Control).Unlike AI Alignment, which seeks to build inherently aligned systems, AI Control develops mechanisms to monitor, evaluate, constrain, verify, and...