%% ### Research strategy - ==**Read actively on a daily basis**: Capture new insights and immediately write them down in the notes section of the relevant note — instead of merely passively consuming information.== - ==**Take the question-centric approach**: Aim to write directly in the research notes and skip intermediary, atomic notes. Create the latter to elaborate on key ideas while writing the research notes if it seems like adding more context would be helpful but more than what's needed for the research note.== - ==**Match sources with questions with the "one to many, then many to one" approach**:== - First, select relevant sources based on questions. Then, study those sources, identify relevant ideas, and write down a reference in research notes to the sources. This is the "one to many" part. - Maybe: Continuously break the research question down into subquestions and subsections as a way to improve the collection of references. - Then, select a research question, and follow the references back to the sources, collect and integrate relevant ideas, and write the answer to the question. This is the "many to one" part. See [[003 Thinking environment]] How can I reduce uncertainty and provide plausible answers to my research questions at the fastest possible rate? Follow Holden's advice in [Learning by Writing](https://www.cold-takes.com/learning-by-writing/): 1. Generally, stay as close as possible to the research questions. Select and study resources with a view to answering those questions. 2. Read 2-3 high-level, shallow books on the topic. Optimize for curiosity and engagement. - Suleyman: [[The Coming Wave]] - Tegmark: Life 3.0 - Russel: Human Compatible - McKee: Uncontrollable - Karnofsky: The Most Important Century 3. Write down (explain and defend) my current best guess in response to each research question. - Also use ChatGPT to search my existing (book) notes and identify relevant ideas. - Maybe create a custom GPT (see [Simmons](https://blockbuster.thoughtleader.school/p/augmented-reading-learn-10x-faster-and-better?utm_source=substack&utm_medium=email&utm_campaign=email-half-post&r=feydb)) that explores any book with a view to answering my core research questions. I could then apply the bot to any new book that I read. - See [this page](https://chat.openai.com/gpts/editor) and [this page](https://help.openai.com/en/articles/8554397-creating-a-gpt) for guidance on how to create a custom GPT 1. Argue against my current best guess. Identify areas of uncertainty that matter. 1. Read hostile book reviews. Scott Young [advises](https://mail.google.com/mail/u/0/#inbox/FMfcgzGxRxBHBnLwfvqcwJwkKhRQztML): "My preferred source for this type of rebuttal is scholarly book reviews. These tend to be written by experts in the field, whereas journalistic book reviews are often written by a non-expert (although this isn’t always true; be sure to check the byline). To find these scholarly reviews, simply go to Google Scholar and type “Name of Book” and “review” to find some examples." 2. Read hostile idea reviews. Young again: "Finding the technical terms that refer to the ideas in the book can help you find critiques of those ideas, even if those rebuttals are not aimed at a particular book. Pretty much any idea you can think of has a name, and once you know its name, you can find people who agree and disagree with it." 2. Select key areas of uncertainty and do more targeted reading around those. Optimize for reducing uncertainty as fast as possible. 3. Revise my current best guess. 4. Repeat steps 3-6 a few times. 5. Seek and incorporate feedback from others. [Metaculus](https://www.metaculus.com/ai/) AI forecasts seem like a great meta resource. It highlights important questions, shows current spread of opinion, and enables discussion among forecasters. Review it regularly. 2024-02-19 reflection: - I started out by reading [[The Coming Wave]] and converting the most important ideas *in the book* into individual notes, then adding references to those notes in my research question notes. This has the advantage of getting the best of what the book offers. However, it suffers from being somewhat disconnected from my research questions. At the end, I will still have to write those notes, pulling together vast amounts of material in distinct notes. - This approach is **idea-centric**: Identify interesting and relevant ideas in the broader space and then flesh those out in individual notes, which then serve as building blocks for various research question notes. - So maybe a better approach is to skip the intermediate step, and directly write my research question notes instead of creating individual ideas notes. The good thing about this is that the ultimate goal is continuously in focus — I'm answering the main questions continuously, rather than veering off into issues that seem interesting but ultimately secondary when answering my questions. - This approach is **question-centric**. - I'm feeling more drawn to the second approach now. It seems more effective overall, but also more effortful maybe. - The question then becomes: **How to match input with output** — i.e., books and articles with research questions? - One approach is **"one to many"**: Study one particular resource, say, a book, identify all the relevant information, then transfer it to each individual research note. - This is more "bottom up": Select great sources, then study them thoroughly, and connect anything that's relevant on the lower level with the questions at a higher level. - Another approach is **"many to one"**: Focus on a single research question, and consult with a broad array of sources that help to provide a comprehensive answer to this particular question. - This is more "top down": Let the research questions on a higher level guide the selection and reading of sources on a lower level. Stay at the higher level as much as possible. - A blended approach would be "**one to many, then many to one**": - First, select relevant sources based on questions. Then, study those sources, identify relevant ideas, and write down a reference in research notes to the sources. This is the "one to many" part. - Then, select a research question, and follow the references back to the sources, collect and integrate relevant ideas, and write the answer to the question. This is the "many to one" part. %% I’m dedicating a portion of my time this year to studying the plausible trajectories and implications of advances in artificial intelligence. I’ve been continuously surprised and impressed by the latest developments and concerned about the potential consequences. My ability to “see around corners” and anticipate what comes next feels very limited at this moment, and I want to invest some time to try and orient myself. >Anyone who isn't confused doesn't really understand the situation. — Edward R. Murrow I break up the topic into a handful of research questions that I aim to answer: %%ChatGPT description of the project: >In the dawn of an era increasingly defined by artificial intelligence, we need to understand and respond to the multifaceted challenges and opportunities presented by AI. This research project is structured to address both empirical and normative questions, focusing on the current state and projected evolution of AI, its broad-ranging impacts across different domains of human activity, and the global risks and opportunities it poses. Simultaneously, it seeks to explore the necessary policies, strategies, and structures that need to be developed in response to these advancements. At the individual level, the project aims to identify the essential mindsets, skills, and knowledge that individuals must acquire to not only thrive professionally but also maintain psychological well-being in an AI-dominated future. By bridging the gap between what is true and what to do, this research endeavors to provide a roadmap for navigating the complex terrain of AI, equipping societies and individuals alike to face an unprecedented technological frontier. %% %% ### Desired outcome 1. Identify and understand the key variables and dynamics that drive change, volatility, and complexity. 2. Understand how to address the emotional challenges arising from ambiguity and uncertainty. 3. Be able to chart plausible paths forward, both over short and long timeframes, for myself and for the world. %% ### Empirical questions — i.e., what is true? - [[What is artificial intelligence and how does it work?]] - This note provides an overview of the technical aspects of artificial intelligence (AI), including its definition, categories of AI systems, key components, and the fundamentals of machine learning and deep learning. - [[What is the current state and projected evolution of AI?]] - This question encompasses understanding where AI technology stands today, including its capabilities, limitations, and the trajectory of its development. This also covers the technological breakthroughs that have led us to the current state and potential future innovations. - [[How will advances in AI affect different domains of human activity?]] - This question explores the impact of AI on various sectors such as the economy, warfare, biotechnology, and societal structures. This includes examining both positive and negative consequences, such as economic shifts due to automation, changes in warfare with autonomous weapons, and the ethical implications in biotechnology. - [[What are the global risks and opportunities posed by advancing AI?]] - Addressing this question involves analyzing how AI intersects and interacts with other global risks (like engineered pathogens or nuclear threats) and opportunities (such as solving complex global challenges). It also involves estimating the likelihood of various scenarios, from best-case to worst-case outcomes. - Meta: [[How much can I trust any of my views on AI?]] - This question explores the reliability and biases inherent in personal perceptions of AI. I'm particularly interested in making sense of the spread of expert opinion on AI issues. ### Normative questions — i.e., what to do? - [[What policies, strategies, and structures should be developed in response to AI advancements?]] - This is about exploring the actions needed to navigate the challenges and opportunities presented by AI. It includes understanding the roles of different stakeholders, assessing what it takes for a positive AI future, and developing institutional strategies to adapt to AI-driven changes. - [[What mindsets, skills, and knowledge are essential for individuals to thrive and maintain psychological well-being in an AI-dominated future?]] - This question aims to explore the set of competencies and adaptive strategies required for individuals to succeed and sustain mental health in a landscape increasingly influenced by artificial intelligence. Furthermore, the question delves into the psychological adjustments and mindset shifts needed to positively embrace the challenges and opportunities presented by rapid AI advancements. ### Bibliography Below is a list of sources that I'm referring to throughout the individual notes on this topic. - Feldman Barrett, L. (2020): Seven And A Half Lessons About The Brain. HarperCollins. Kindle Edition. - Hendrycks, D. (forthcoming): Introduction to AI Safety, Ethics and Society. Taylor & Francis. URL: [www.aisafetybook.com](http://www.aisafetybook.com/) - Hendrycks, D. et al. (2023): An Overview of Catastrophic AI Risks. [arXiv: 2306.12001](https://arxiv.org/abs/2306.12001). - Nielsen, M. (2024): "Notes on Differential Technological Development", [https://michaelnotebook.com/dtd/index.html](https://michaelnotebook.com/dtd/index.html) - Russell, S. (2019): Human Compatible. Penguin Books Ltd. Kindle Edition. - Suleyman, M. (2023): The Coming Wave. Crown. Kindle Edition. %% #### Archived questions - The most important part is asking the right questions. - Metaculus has a [list](https://www.metaculus.com/ai/) of AI-related questions and corresponding forecasts. - Where are we right now? How did we get here? What's the current frontier? - I'd have to come up with a set of key questions again that guide my learning and force me to articulate my insights and uncertainties. - What are the domains that AI will touch upon? - Warfare: autonomous weapons - Economy: drive down cost, replace people with machines - Biotechnology: engineered pathogens - Society: trust between people, trust in institutions - What will be the incentives that shape the situation? - E.g., Moloch, individuals seeking status and legacy, etc. - What are the potential good and bad outcomes? What are the expected outcomes? - What’s the economic, political, and cultural environment in which this will play out? How will this affect things? Eg polarization - What if AI safety is a social science problem? - How likely is an intelligence explosion in the next decade? What would it look like if it happened? - What is AI alignment? How hard should we expect alignment to be? - What would it take for this to go well? (Cf Your Undivided Attention podcast) - How might advances in AI interact with other global catastrophic or existential risks? - E.g., AI might catalyze new bio or nuclear risks - Who is working on this? How likely are they to succeed? Are there any adults in the room? - What will AI be able to do by when? Cognitive labor? Manual labor? What will be the impact of that? - Estimate likelihoods for different scenarios? What's merely possible, what's plausible, what's even probable? - What are the best arguments against TAI happening a) at all, b) soon (i.e., in the next 5-10 years)? - Maybe there will be unforeseen bottlenecks that slow down or even prevent further progress - Data bottlenecks? - Further theoretical breakthroughs as requirements - AI systems can't generalize beyond their training - Given my predictions, how do I want to act? How do I want to manage myself? What do I need? - Take stock at the outset. What do I believe at the start of the year? How do I feel? - How do I achieve equanimity in the face of all this? - How do you cope with extreme volatility and uncertainty? VUCA on steroids. - What does all of this mean for my career? What are some paths I could take? - What does all of this mean for my life? ### Sources See [this spreadsheet](https://docs.google.com/spreadsheets/d/1Lh8-YXAPE4PtYKehpVM3c8uBG2SzP34M61-rARXdr98/edit#gid=0) ### Glossary Maybe add a section that explains the key terms that I use throughout these notes? For inspiration, see "Glossary" in [[A Brief History of Intelligence]], or "Glossary of Key Terms" in [[The Coming Wave]]. #### ChatGPT on how to organize sources **My takeaway**: Create a spreadsheet and add all of the previous sources. Use all of the perspectives offered below as distinct columns in the spreadsheet. See Jamie Harris' [example](https://docs.google.com/spreadsheets/d/1e8Ont6TT8t06qQb16AYIJlSdV-_EaZsoD-Cw57BBVBc/edit#gid=0) for inspiration. Organizing and categorizing your sources is crucial for efficient research and analysis. Considering the diversity of your sources, I suggest a multi-dimensional categorization framework. Here’s a structured approach: 1. By Content Type: - **Books:** For comprehensive, in-depth analysis (e.g., "The Precipice" by Toby Ord, "Superintelligence" by Nick Bostrom). - **Academic Papers and Reports:** For data-driven, scientific insights (e.g., Hendrycks et al., IMF study on AI and labor market). - **Blogs and Newsletters:** For current opinions, trends, and discussions (e.g., Holden Karnofsky's blog, Vitalik Buterin’s techno optimism). - **Podcasts and Interviews:** For expert opinions and discussions (e.g., 80k podcast, Dwarkesh podcast). - **Online Forums and Communities:** For community-driven insights and debates (e.g., Metaculus, LessWrong content). - **Videos and Documentaries:** For visual and auditory learning (e.g., Robert Miles YouTube channel, David Shapiro YouTube channel). 2. By Topic Focus: - **AI Development and Progress:** Covering the technical and theoretical aspects of AI development (e.g., "Artificial Intelligence" by Melanie Mitchell, Andrew Ng’s talks). - **Societal and Ethical Implications:** Focusing on the social, ethical, and philosophical aspects (e.g., "Human Compatible" by Stuart Russell, “God Human Animal Machine” by Meghan O'Giblyn). - **Economic and Political Impact:** Addressing the economic, political, and global implications (e.g., "Principles for Dealing with the Changing World Order" by Ray Dalio, Economist articles). - **AI Safety and Risk Management:** Concentrating on the risks and safety measures related to AI (e.g., AI Safety Fundamentals, Jessicata’s posts on AI alignment difficulty). - **Future Scenarios and Predictions:** Exploring potential futures and forecasts (e.g., "Age of Em" by Robin Hanson, “Chip War” by Chris Miller). - **Personal and Professional Development:** Guides and advice for individuals adapting to an AI future (e.g., Nielsen’s notes, Grace’s counterarguments to AI risk). 3. By Perspective: - **Optimistic/Pro-AI Views:** Sources that present a positive or progressive view of AI (e.g., “AI 2041” by Lee, “The Age of AI” by Kissinger et al.). - **Critical/Skeptical Views:** Sources that critically examine AI development and its implications (e.g., “Rebooting AI” by Gary Marcus, Nuno’s skepticism braindump). - **Balanced/Neutral Analysis:** Works that aim to provide a balanced overview (e.g., “The Alignment Problem” by Brian Christian, Benaich’s State of AI Report 2023). 4. By Date of Publication: - For keeping track of evolving thoughts and technologies over time. Categorize the sources based on their publication year or the period they cover. 5. By Author or Publisher: - Grouping sources by author or publisher can help identify unique perspectives and biases. For example, works by a particular thought leader or outputs from specific institutions or think tanks. Additional Tips: - **Cross-Referencing:** Some sources may fit into multiple categories. Maintain flexibility in your framework to cross-reference them as needed. - **Annotation and Summarization:** For each source, keep a brief summary or annotation to recall the key points and why it’s relevant to your research. - **Digital Tools:** Utilize digital tools like reference management software (e.g., Zotero, Mendeley) to organize and easily retrieve your sources. This structured approach should help you navigate your resources effectively, allowing for a comprehensive and nuanced understanding of the various facets of AI and its implications. ### Notes - Vishal on 2024-01-11: "There may be funding booms and bust cycles as assets are mispriced and re-priced while volatile macro forces play out, but we believe the fundamental forces driving AI progress (primarily hardware improvements) have been consistently intact for decades, and we have no reason to believe that they will stop. Scaling laws remain intact, private and public sector actors are deploying exponentially more resources into AI, and we are starting to see the beginnings of recursive self-improvement of AI software and hardware. As AI becomes increasingly integral to the AI R&D process and business operations, we expect teams to operate with increasingly machine-driven workforces. Several founders we’ve spoken with are already planning their headcount and product development roadmaps around this." - AI capabilities develop at a faster pace than our ability to comprehend them. - We're on an exponential curve in terms of tech development. Our intuitions lead us astray in this environment. Imagine predicting the level of tech of 2023 back in 2020. Now try again for 2026 in 2023. You'll probably be off by a lot. - Apparently Ajeya Cotra uses this framing: AI tech is a bit like 24th century tech crashing down on the 21st century. Compare it with 21st century tech dropping on the 16th century or so. - Apparently Charlie Munger said something along the lines of: If you show me the incentives at play, I'll show you the outcome you'll get. If you know the race, you know the result. - Holden argument on the 80k podcast: If AI will develop very fast, we likely won't be able to adapt to it and solve problems as they arise. Usually, this is the right approach – wait until something arises, then diagnose the situation and take steps to mitigate it – but in the case of AI, the prospect of very fast improvements makes this hard to pull off. - Start with humility. Our ability to comprehend and model where we're headed is severely limited — at least if the assumption is true that the future will be radically different; that there will be radical acceleration in growth etc. We're too parochial to understand. - [[Introduction to Generative AI]] - Mythos thesis: TAI may arrive within a decade. - Scaling: The success of scaling (increasing compute, data, and model size) suggests rapid, predictable research progress in the years to come; capability improvements need not be bottlenecked by unsolved theoretical problems. - Recursion: AI has become increasingly effective at essential components of ML research. This sets the stage for recursive improvement, which can rapidly accelerate capabilities. - Resourcing: The amount of money and brainpower funneled towards AI research has skyrocketed, and will likely continue to rise. This increases the likelihood of theoretical breakthroughs and the speed of the research-implementation transition. - ![[Pasted image 20231116111915.png]] - Amara’s Law: “We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.” Social change is slower than technological change. We should not expect to see immediate global effects of AI in a major way, no matter how fast its adoption (and it is remarkably fast), yet we certainly will see it sooner than many people think. (From Mollick's [Substack](https://mail.google.com/mail/u/0/#inbox/FMfcgzGwJcdTPbgxgTZVkbVHxSSzfQrL)) - William Gibson, who famously wrote “The future is already here - it is just unevenly distributed” - a point backed up by decades of research on user innovation. Also from Mollick's Substack. - 2024-01-13 Yuval Noah Harari on the Steven Bartlett podcast: AI is a fundamentally different technology from, say, writing or the printing press. It's the first technology that can make decisions on its own, and that can create new ideas. - The Wait Calculation (from Mollick's [newsletter](https://mail.google.com/mail/u/0/#inbox/FMfcgzGwJmFWbKdSFdLrTXhDfVVZKKlb)): If you consider investing in any task or project, you should consider what the right time to do so will be. Given the rapid exponential development of AI, waiting rather than doing now might often be the right response. - Mollick: "There are really two questions that matter when it comes to the Wait Calculation for AI: How good? And how fast? In other words, _how good_ can this particular run of AI technology (either LLMs or their immediate successors) get in terms of human achievement; and _how fast_ will this happen?" - Here is what anyone considering a project in an AI-adjacent field should consider: - How long will the project realistically take to complete? - Given timelines for AI development, how much more advanced could AI get by that point? - How much time could you save by using the AIs developed between now and the end of the project? - Will the AI just be able to do the thing before the end of the project? Or will the meaning of the thing shift? - Mollick: "No one actually knows the answer, of course, and, in some ways, it is a particularly awkward time to predict the future of AI. Despite the massive advances in the field, GPT-4 remains the world’s best AI, over a year after its preliminary release. A large number of companies are building AI models that they say will surpass it soon, and OpenAI itself keeps hinting broadly at its own upcoming releases. We just haven’t seen them yet." - "Another way to look at the _how good_ and _how fast_ questions is to look at prediction markets, where people bet on the dates that particular events will occur. These are often surprisingly accurate, and one of the key markets puts the date of [“weak” AGI](https://substack.com/redirect/e8e43d4e-5db2-4a5f-9c61-c965ae9589da?j=eyJ1IjoiZmV5ZGIifQ.ZinwQ5gMH8ZmxCs3tQHE_efEnwOGpp9U1XRcdU3Pe28) (at the 75th percentile of human abilities across subjects) as happening in a little over two years, and [full AGI](https://substack.com/redirect/c2f95042-ea30-481b-98f9-35964ef37760?j=eyJ1IjoiZmV5ZGIifQ.ZinwQ5gMH8ZmxCs3tQHE_efEnwOGpp9U1XRcdU3Pe28) within a decade." %% --- Topics: - [[Artificial Intelligence]] - [[Existential Risk]] - [[Futurism]] Related notes: - [[How to work with emerging AI tools]]