# [Computing power is now doubling every 6 months, as the Scaling Laws paradigm has taken over from Moore's Law, and the new currency is tokens per dollar per watt](https://x.com/tsarnick/status/1848463329972654104)  *** <iframe title="Satya Nadella AI Tour Keynote: London - YouTube" src="https://www.youtube.com/embed/kOkDTvsUuWA?feature=oembed" height="113" width="200" allowfullscreen="" allow="fullscreen" style="aspect-ratio: 1.76991 / 1; width: 100%; height: 100%;"></iframe> #### [[👤Satya Nadella]]: The scaling laws in AI are empirical, similar to how Moore’s law was. Moore's law wasn’t a physical law, just an observation that held true. The scaling laws show that about every six months, we see a doubling of capacity. Performance is now measured in terms of tokens per dollar per watt, and this metric doubles every six months. This improvement comes partly from more compute power, but much of it comes from better ways to use data and algorithms. This trend began with deep neural networks (DNNs) in the early 2010s and accelerated in 2018-2019 with the rise of large language models (LLMs) and transformers, and it continues today. #🤖Satya_Nadella *** - [[The cost per million tokens has fallen from $36 to $0.25 in the past 18 months, and as such AI is the greatest cost-deprecating technology ever invented]] - [[While scaling compute is the most trustworthy way to improve AI models, they have also become 100x more parameter efficient in the past 18 months, resulting in a "surreal" rate of progress]] - [[Algorithmic advances in AI in recent years are outpacing the increased compute that's put into the models]] - [[AI compute is increasing "hyper-exponentially", growing at a rate of about 500% per year, and in addition the algorithms are becoming more sophisticated, so it is improving both quantitatively and qualitatively]] - [[Just as the camera did not end painting and art, AI will not be the end of creativity, but instead it will empower artists and writers to create different and incredible things]] - [[Energy and intelligence are the 2 key inputs to techno-abundance and eventually the cost of intelligence will approximate the cost of energy, so we need to invest in energy solutions as the biggest constraint on the cost of AI]] - [[in the past 85 years, computing power has increased by 75 quadrillion times for the same cost]] - [[Technology has now reached a positive feedback loop where AI is designing new AI and is now advancing at the pace of "Moore's Law squared", meaning that the progress we will see in the next year or two will be "spectacular and surprising"]] - [[The cost per query in AI models has decreased by 100x in the past 2 years and quality will improve as hallucinations decrease 10x per year]] - [[The cost of inference compute will reduce by 100x in the next 2 years due to the scaling law]]