~www_lesswrong_com | Bookmarks (659)

Published on July 26, 2024 12:33 AM GMTA Solomonoff inductor walks into a bar in a foreign land. (Stop me if you’ve heard this one before.) The bartender, who is also a Solomonoff inductor, asks “What’ll it be?”. The customer looks around at what the other patrons are having, points to an unfamiliar drink, and says “One of those, please.”. The bartender points to...

Published on July 25, 2024 10:03 PM GMTStatus: Thought experiment for fun Imagine a world in which the gambler's fallacy is fundamentally true. Functionally, lets suppose there's a magical force that tracks a thinking being's expectation of any particular outcome, and then mysteriously increases the likelihood of said outcome the more often it had physically plausible opportunity to occur and did not[1]. (This resets upon...

Published on July 25, 2024 10:00 PM GMTWork done at FAR AI.There has been a lot of conceptual work on mesa-optimizers: neural networks that develop internal goals that may differ from their training objectives (the inner alignment problem). There is an abundance of good ideas for empirical work (find search in a NN, interpret it), but very little actual .mjx-chtml {display: inline-block; line-height: 0; text-indent: 0;...

Published on July 25, 2024 8:55 PM GMTAdversarial vulnerabilities have long been an issue in various ML systems. Large language models (LLMs) are no exception, suffering from issues such as jailbreaks: adversarial prompts that bypass model safeguards. At the same time, scale has led to remarkable advances in the capabilities of LLMs, leading us to ask: to what extent can scale help solve robustness?...

Published on July 25, 2024 7:11 PM GMT2024-07-25: I'm organising a little reading group, undecided between in-person in London or remote. My own questions I want to get the answers to are in https://www.patrickstevens.co.uk/posts/2024-07-25-lob-theorem/ . Writing this here so we have a place to start coordinating how we do this. I'm away on holiday until next Sunday so won't be doing anything about the...

Published on July 25, 2024 2:58 PM GMTThis is based on our recent preprint paper “Constructing Benchmarks and Interventions for Combating Hallucinations in LLMs” by Adi Simhi, Jonathan Herzig, Idan Szpektor, and Yonatan Belinkov.Code - https://github.com/technion-cs-nlp/hallucination-mitigationSummaryIn a question-answering (QA) setup, hallucinations can occur when a model generates incorrect answers.Three types of hallucinations can be identified based on the model's knowledge of the correct (golden)...

Published on July 25, 2024 3:58 PM GMTGoogle DeepMind reports on a system for solving mathematical problems that allegedly is able to give complete solutions to four of the six problems on the 2024 IMO, putting it near the top of the silver-medal category.Well, actually, two systems for solving mathematical problems: AlphaProof, which is more general-purpose, and AlphaGeometry, which is specifically for geometry problems....

Published on July 25, 2024 3:54 PM GMTThere doesn't seem to be an arxiv PDF out yet for AlphaProof, so I'm linking the announcement post. Relevant quote: AlphaProof is a system that trains itself to prove mathematical statements in the formal language Lean. It couples a pre-trained language model with the AlphaZero reinforcement learning algorithm, which previously taught itself how to master the games...

Published on July 25, 2024 3:49 PM GMTThis is an edited transcription of the final presentation I gave for the AI safety camp cohort of early 2024. It describes some of what the project is aiming for, and some motivation. Here's a link to the slides. See this post for a more detailed and technical overview of the problem.This is the presentation for the...

Published on July 25, 2024 1:50 PM GMTWe got two big model releases this week. GPT-4o Mini is covered here. Llama 3.1-405B (and 70B and 8B) is mostly covered in yesterday’s post, this has some follow up. Table of Contents Introduction. Table of Contents. Language Models Offer Mundane Utility. All your coding are belong to us. Language Models Don’t Offer Mundane Utility. Math is...

Published on July 25, 2024 11:18 AM GMTSammy Martin, Polaris VenturesAs AI systems become more integrated into society, we face potential societal-scale risks that current regulations fail to address. These risks include cooperation failures, structural failures from opaque decision-making, and AI-enabled totalitarian control. We propose enhancing LLM-based AI Constitutions and Model Specifications to mitigate these risks by implementing specific behaviours aimed at improving AI...