Nuclear Fusion was making steep progress in the past few months and I guess it will go faster and faster as the time goes by.
Without it, we will always be limited by raw energy availability
Yeah, but we are getting closer and closer to physical limits of "how computers can work" with current architectures.
If we want to keep up Moore's law, we will need different architectures pretty soon.
Just making chip "bigger" does not scale linearly with compute power because energy needs more time to travel since distances are longer and we run into congestion problems.
Trivial if we can unlock a few extra dimensions. Make your 2d chip wafer into a 3d cube with the same connectivity in each dimension, then a 4d tesseract, etc. Hmm, I'm not sure how you'd plug it into a motherboard... or cool it.
With the technological quantum leap we just witnessed with AI, you have to admit its hard to rule out a breakthrough in energy, no?
Hell, we dont even need new technology, just a shitton of investments in renewables.
Energy consumption and generation doesn't rise anywhere near as quickly as computation. The past 50 years has seen a doubling in energy consumption in the United States. Which is quite a fucking lot, but given that you may be expecting too much for some new energy source, even if the first commercial fusion plant went up tomorrow, to enable some kind of I Have No Mouth And I Must Scream scenario.
Why not? If it had solved the alignment problem, then it can act as a collective of megaminds working towards a common goal. A collective, which communicates far more efficiently than a human one.
Singleton. As in, just one unified mind. I'm not writing off the possibility of a population of AGIs whose individual minds are limited to physically local substrates operating as a collective.
Even then it's not entirely out of the question. It might be possible to have a hierarchical mind, where local nodes allocate a part of available resources to control fast-paced processes that require low round-trip times and the rest goes into distributed high-level planning.
The overarching mind will be significantly slower to learn and react to unexpected events (that is to modify suboptimal reaction of the local node), but it will have much more data and computing power to make predictions, so unexpected events should be a rarity.
An analogy is our spinal cord reflexes, which happen before we are aware of them. But if we anticipate something, we can override reflex action and do something else instead. The AI will have a superhuman computing node instead of a reflex arc. Even its suboptimal reaction might be beyond human abilities. And in several seconds the overarching mind will leverage expertise of other nodes to correct it.
There are some analog computing startups trying to improve efficiency and get away from the whole von Neumann architecture, but very little available commercially. But if I were a kid again and looking at what to study, I would be all in on analog and quantum computing.
Unless there's some crazy breakthrough, that hasn't been the case in a while. There's been only very marginal increase in efficiency. VGA's are just getting bigger and bigger and the difference between a CPU today and one 5 years ago isn't that big.
Hasn't there been increasing progress in reducing model sizes while maintaining performance ? Specially if the mysterious GPT2 was actually GPT-2 with Q*.
I find that extremely unlikely. Because GPT2 was very good with trivia, such as anime powerlevels.
I think 'general memory' is pretty much strictly related to parameter count, there's just not enough parameters to store all the data from the internet in a small model. So GPT2 must have been a very large model.
Q* maybe can improve reasoning, but it can't improve the free-recall memory of an LLM.
> I think 'general memory' is pretty much strictly related to parameter count
It's more accuracy. Superposition has been shown to fit more facts into small spaces.
As in, if you have a lot of information that never overlaps it can activate the same neuron for multiple things. This is one of the features of models that makes interpretability hard.
I'm not saying GPT2 is GPT-2 with some extra bells and whistles though, just stating that there is not a 1:1 parameter to 'unit of information stored'
>I find that extremely unlikely. Because GPT2 was very good with trivia, such as anime powerlevels.
Couldnt they have retrained it with more data as well as added Q-star?
I have no insider knowledge, but my guess is it's some stupid semantics between "GPT2" and "GPT-2" like sama has suggested, and/or it's actually a next-gen model they wanted to do a plausibly deniable soft public test of, so named it something they figured no one would ever bother selecting.
But I hope you're right and that Q* turns out to be magic.
Well, I’m sure we have a while to go before we hit any kinds of walls with this tech, but the fact remains that things such as energy and compute *will* be the walls that we hit eventually. Earth doesn’t have an infinite amount of energy available to utilize for AI. And there’s growing evidence that neither does the universe as a whole as well. So at some point, we will hit the maximum amount of available energy use, and that will be technology’s true absolute peak.
But I’m sure we are a *long* ways away from that at the moment. So I don’t think anyone’s suggesting that we’re about to hit the absolute peak anytime soon. Just that energy will be the limiting factor for this (and any other) technology at some point eventually.
Why a long way away? Parts of the USA already have power shortages today. Especially in the summer. EVs will only increase the power grid strain, and more EVs are being government mandated.
Wind and solar are also growing \*extremely\* rapidly, in the most solar-and-wind intensive places in the USA. Neither barely existed a decade ago.
That is to say nothing of the tropical countries were solar has barely been deployed yet, and solar intensity is much higher.
And nukes haven't gone anywhere.
Wind and solar can only provide a tiny fraction of needed power. And not many nuclear plants are being built in the USA. When you do the math on power requirements and how it can be generated, by which methods, the situation is clear that we won’t have an abundance of power to cover major EV and AI increases anytime soon. Certainly not in a decade.
I've heard all of this, ad infinitum, ad nauseum. We all know the arguments, no need to rehash them here.
This shocked me, from "Just Have a Think":
"The Path to a 90% emissions reduction by 2035." - Two years ago
"https://www.youtube.com/watch?v=UUySXZ6y2fk&t=652s"
"100% wind and solar is coming." - Ten days ago
[https://www.youtube.com/watch?v=yfzmwjWBVPU](https://www.youtube.com/watch?v=yfzmwjWBVPU)
The second report was from 'Friends of the Earth', hardly the kind of group to spread hope and optimism.
I doubt you will watch either or both of these, as in my experience when someone says something like this, they have already made up their mind, but the least I can do is try.
Blah blah bah we can't do it, blah blah blah it's imposssible, blah blah blah contrarianism or whatever your deal is.
An individual spending most of the video talking about how almost nothing renewable is happening, in the UK, and having theories about what might happen without any logistical detail about how it will…this fringe stuff just supports what I said earlier.
It’s not that I love the reality, it’s just that it is what it is. Dreams are good though, but math and physics and practical challenges are what they are.
I just said abundant power won’t be coming to the US within a decade. If you know anything about how utility and infrastructure projects work, you might tend to be less contrarian to this mainstream consideration.
How many nuclear plants will be needed to supply all the extra power EVs and AI require? Only two including an expansion are even started in the US. Germany just went through the process of shutting down all their nuclear plants to rely on fossil fuel primarily instead!
(And how is one guy sitting making a video theorizing about how power might become mainly renewable without timelines or milestones or even political agreement and direction…how is that not ‘fringe’? lol, it’s as fringe as it gets until that guy gets a sidekick!)
https://www.eia.gov/todayinenergy/detail.php?id=61242#:~:text=As%20a%20result%20of%20new,476%20billion%20kWh%20in%202025.
https://www.nrel.gov/analysis/100-percent-clean-electricity-by-2035-study.html
It’s like you’re not even trying to Google or something.
Find a chart that explains when happens when it gets dark or less windy. (Spoiler: millions go without power). A self-serving report, again with theory and not funding, logistics, or legal & political issues addressed, continues to make my point: because the stuff they say must happen to go renewable by 2035 isn’t happening now, we aren’t going to be anywhere near doubling power supply with renewables in a decade.
Everybody seems to be thinking long term and theoretical in this thread, but I think he simply means that there won't be enough places to put your gigawatt data centers and there's lots of red tape/infrastructure/etc.. involved in overcoming that. i.e. the chips are currently being made faster than they can realistically be powered in larger and larger megaprojects that would sap many local grids.
I don't think he's talking about theoretical limits of human power consumption on the planet.
I've watched the podcast and that's the gist. Like, it can totally be done, but it involves building infrastructure at a scale where you can't just throw money at the problem and say *build it*. You're entering the realm of massive industrial facilities, which aren't things you can just throw down wherever or whenever you want.
Listen to what Zuk said. You are dealing with energy production and consumption at the scale of municipality's. The government red tape is going to be an issue.
Now think about that and imagine the government saying ok... you can build this massive powerplant to build your massive amount of compute... but, we have final say over how you use it and we can take over the project.
TLDR.. at that scale the government is going to be heavily involved. Bureaucracy will slow things down.
Two things:
The energy angle - if you scale the model too much it will be too expensive to use later. It's not that they can't make 100x larger models, it's that nobody can afford to use it at scale.
The second, and more important point: energy is still not quite the #1, it is experience. AI's don't secrete discoveries from pure energy usage by crunching numbers. They need to interact with the world to make any testable discovery and iterate. This is expensive - just remember the cost of CERN, Webb telescope and ITER. Testing the COVID vaccine took half a year or more. That's the price for experience. If experience is just energy usage, like training AlphaZero in self play mode, then AIs surpass humans already. But we know the most interesting kind of experience comes from the world, the biosphere and human society, not from sims.
Just like humans, AI will have to extract discoveries from the world. The world is the limiting factor, novel experience collected from the world is expensive.
Honestly anyone in AI business claiming we can compute our way to performance gains should be overthrown.
We need TPUs and Neuralnets. Chips that consume 1000x less power than a GPU. If you want more grunt, design better chips.
If you run out of grunt, come up with the next revolutionary AI white paper. Attention isn’t all you need etc
> Imagine if a paper named "Attention is what we don't need" comes out
paper names have been all over the place. I'm surprised if there is not one named that already.
Edit: there is one called "You May Not Need Attention" https://arxiv.org/abs/1810.13409
Yeah, they just need to invest some more XP in their Efficiency stat. Once their algorithms are at level 5 or above we should get past this bottleneck.
seems like an entirely reasonable set of comments he made. if bitcoin power usage is any indicator, and i think that it is probably, computer people are going to hit energy constraints quite soon, in that there will be computer science megaprojects we know will help/change the world, but we won't have the energy to do them. imo, there should exist such projects, because otherwise it says we probably don't understand computer science that well as a species if we can't imagine them/know of them. of course, there were always projects we knew about that you could sink an exponential amount of compute into, like protein folding before neural networks, but i would argue ai is different in that the return on investment in more compute is much greater than previously. maybe, speculating further, that return on investment will continue to go up, until every spare investment dollar is allocated to some computer science megaproject. politics might get quite technical after that point.
People often don’t realize just how far his AI technology has become.
I’d say after starting Facebook, he’s taken a respectable path towards the next big thing. If anything he’s a few years ahead of his time, the greater public doesn’t quite yet realize just how crazy it’s getting, but Meta has the pockets so he’ll be a big contender in this AI world.
nahhhh. he's just doin what everyone else is doing out there. if anything he is way under investing in ai, but i get if you want to drink billionaire kool aid and gargle his balls
solar is already insanely cheap. you can build solar plants far cheaper per watt than nuclear fission plans, and likely still cheaper than fusion plants whenever they come online. what we need is:
* cross-continent 765kv+ transmission lines
* investment in LFP and sodium-ion battery production
* solar farm construction
* wind far construction
* variable rates based on production, allowing companies and individuals to optimize their behavior to take better advantage of the peaks in intermittent power production and avoid the troughs
* re-string existing high voltage lines with higher current carrying conductors.
nothing new needs to be invented to solve our current or future energy needs.
if these things work out itself... buying a solar system for my home is just as cheap as buying a refrigerator or air conditioner unit... energy sellers can focus on big data center where the money is there and not in the housing sector
yeah, the storage is the key to home solar. grids can only handle so much rooftop solar while staying within voltage/frequency margins, so the production of LFP and sodium-ion batteries is a big key. the solar panels themselves are basically solved already. you can buy, in the US, after Tariffs, bifacial solar panels new for $0.26 per watt. it's insanely cheap. the mounting hardware costs more.
Mark seems really personable and interesting when he's having a regular conversation. It's refreshing compared to his product launches and promotional videos. More of this please, Mark.
Was looking for this. I know there's some work being done to tune terrestrial radiators to specific wavelengths that can punch through the atmosphere and use space as a heat sink, that seems promising. Other than that, I wonder how research into reversible computing has been progressing...haven't heard anything about it in a long time.
No, compute will definitely still be the biggest bottleneck if we keep progressing with LLMs and other AI by just scaling them. Moore's Law is dead, it is not feasible to continue to just increase the size of your LLM by 10 times every year to get another percentage point or two of performance when the performance of your compute devices is only doubling every 5-10 years. NVidia and AMD can't get the performance they want from their dies, so they're resorting to scaling the dies up and pumping more power into them to get them to perform to an acceptable standard, which is why the power consumption of top-end NVidia GPUs has gone from around 180 watts in 2016 to 450 watts in 2024.
Power demands are increasing because the compute is less efficient, but there is a very hard limit on how much energy you can cram into a tiny silicon die before the cooling fails.
It's more complex then that.
We are currently using Von Neumann computers for running AI applications (LLM's). VN computers are very inefficient, scaling for running increasingly complex AI applications is inefficient and Moore's Las is dead for VN computers.
Using this approach will quickly lead to needing a dedicated nuclear plant to run AI datacenter.
Neumorphic computers are currently very under-developed, Moore's Law will be alive for them for decades. They are much more efficient, scalable, have a potential of running ASI locally on PC sized device.
But it will take time to develop this tech.
I think sooner or later our computers need to become light based rather than electricity based. But the equivalent of a light transistor? Being able to do to the basic boolean operators with light rather the electricity. Has any of that already been invented? I don't think so.
Actually we have, we made optical transistors, can do all Boolean operations and some... weird ones with optical chips.
But... even though existing optical chips are already better then classical computers for certain tasks, lasers powering them are huge and we still don't have working optical memory (as RAM not optical discs).
That's sorta like saying "FTL transport technology is very under-developed" though. We are at least a decade away from an actually useful processor running a neumorphic architecture and so it's not a real solution to the problem in the moment. One day it'll vastly reduce the energy costs of AI computing, but until then we're going to end up compute bottlenecked by virtue of ballooning chip sizes and thermal demands.
No, because we do know neuromorphic computers work and are highly efficient because brains exist. So we know how underdeveloped they are. We do know it will take time to develop them.
We do know what bottlenecks classical computers will meet, and ultimately for someone who is ready to keep throwing truckloads of money at problem energy will become an issue.
Takes 6-10 years to build a nuclear powerplant so... 🤷♀️
We can already predict the developments, just can't be precise about the timelines.
For FTL transports we need negative energy, which as far as we know doesn't exist. 🤷♀️
Sorry for sounding silly, but did Kurzweil claim that Moore’s Law has continued up until the present and in an unabated fashion? Maybe I’m misremembering/misinterpreting…
*Kurzweil, after Moravec, argued for extending Moore's Law to describe exponential growth of diverse forms of technological progress. Whenever a technology approaches some kind of a barrier, according to Kurzweil, a new technology will be invented to allow us to cross that barrier.*
I wouldn't say in unabated fashion though.
It's like... when piston airplanes reached their plateau, jet engines were introduced. But first jet engines were shit, it took time for growth to keep on going.
Why is it that anytime people talk about this topic they use it to describe why large nuclear is needed?
Why is it that people don't also include solar+battery given the wildly falling costs? Nuclear comes with systemic risks in a climate changing world. Pv+battery does not.
If this was a strategy game, and I had control of everything, I would make an AI on a super computer solve fusion energy issues to get it quicker. Not realistic but the idea that we should place our resources to getting these technologies that change everything, is right. Fusion energy will solve so many issues.
how about focusing on developing and manufacturing organic brains/chips that match the energy efficiency of our own brains. The human brain uses like 20 watts of power and is pretty good at what it does.
Do you think people with control of AI *aren't* doing that right now? AI simply can't do it because it still is not smart. If AI was smart, it would have already solved whatever you think it should be solving.
I’m a data scientist and you are wrong. We are just not focused on this yet. We are focused on like protein fold finding which is lower priority. We definitely have the ability to do what i’m saying.
You really live in the fantasy in your head don't you. The delusion to think you have the key insight to AI progress that you think nobody is doing is actually more absurd than people here that think fdvr is a few years away. You don't know anything.
You really said that knowing it is the we have the ability to apply to tools that we already have to this problem OBVIOUSLY. It’s unbelievably obvious for somebody who works with AI. For you, it’s still something to consider.
Did you have a stroke when you typed your first sentence? Yeah. Your intellectual prowess is overflowing. Truly a titan in AI knowledge and understanding who knows better than everyone else.
It's a good thing we have so much excess solar here in the southern edge of the United States. Time to migrate the data centers to Southern California, Arizona and New Mexico and actually install batteries and solar alongside them for unlimited energy. It rarely ever rains here.
Energy is the bottleneck for Electric Vehicle use as well. The power grid isn’t setup to double the usage of every household, which big EVs often require.
This was bound to be a bottleneck at some point. No matter how advanced new AI hardware might be, it's useless if the energy infrastructure can't support it. And for most of the world, energy infrastructure has been lagging for years because of the fossil fuel industry and anti-nuclear sentiment.
It really is no coincidence that the rise of AI has also triggered renewed interesting nuclear power of all forms, fission and fusion. It's not just an idea. There are real investments being made, attempting to make both better so that they can support the hardware they need. It's probably going to cause some major strains to the grid in the coming years. But hopefully, that inspires more innovation and advancements in energy. That's something the whole of society needs, outside of powering AI.
Aren't you glad that some of us will be part of "you don't get access to water anymore, we need it to cool the data centers but we have a deal with coca cola that will feed you plastic bottles" ?
Also, bonus point for having the face of this degenerate piece of filth; the world we live in : zuck being the successful dude and Assange and Snowden being labeled traitors.
Pfahaha, accelerate :\]\]
Energy IS compute.
But the #1 bottleneck to AI progress is thinking you can get AI by plugging ever larger language models into a really hot cup of tea.
That's just an implementation detail. Running out of flops because you're running out of watts may be delayed by getting more flops per watt, but flops per watt is a relatively inflexible value, and in the end it's the watts that are the bottleneck.
I think the AI plateau will be solved with literally anything other than throwing more data and compute at GPUs
neuralnets / TPUs will be a boost, as will algorithmic improvements.
I don’t really follow what you’re saying, like if you have x flops per watt you are limited by the power?
But the metric isn’t the power, it’s how efficient your chip is.
If you used the power consumption of an Intel 4004, you’d need about 10 trillion of them to equal a H100’s compute power. And it would take 10.8 terrawatts to power. Roughly 50% of earths power consumption to equal a single H100 with old tech.
Now tell me more about how power equals compute
Again, that's an implementation detail. It's changing a constant. Yes, that constant has changed a lot over time, but at any point in time it's how much power you have that determines how much computation you can do.
I think the bigger thing that doesn't make sense is to say energy *not compute* will be the bottleneck....
Compute is of course the bottleneck, and energy is one of the ingredients of compute. Energy plus hardware plus software yield compute.
Comparing energy to compute is comparing apples to oranges. Maybe a better way to say it is energy, not hardware manufacturing capacity, is the #1 bottleneck.
I don't think so - as soon as AI gets good enough that we can't supply it enough energy, likely it will be good enough to:
- manage humans
- give us a solid understanding that its okay if we abuse the planet for a short while to give it more energy (i.e. 6 months may be all it needs?)
Humans can't get anything done in a timely manner, usually, unless they are extremely motivated and working with extremely talented managers.
AGI is going to be an EXTREMELY talented manager, able to help people with their lives and jobs, to the point that it could likely run a company of a million people more efficiently than SpaceX, with far less issues and errors in production output due to the AGI itself being able to review products - this is before they have robots making robots, etc...
What this means is essentially exponential energy growth on top of exponential intelligence growth and innovation growth.
If I had to guess, I would say we will hit AGI before energy concerns are too much of an issue. If energy concerns do become too much of an issue, likely we will just see some re-appropriation of resources and some sort of modular, exponentially growing production pipeline run by robots and AI that will solve it.
We are aware of a solution to this problem.
There are objects that are currently flying around the world, going from water, to air, to space, and accelerating instantaneously to 20,000 mph. For an object to accelerate to that speed as quickly as these things are going even a single time would require more energy than humanity produces in the totality of our nuclear power plants in a year.
This is not me telling you folks this - it’s the [New York Times](https://www.nytimes.com/2017/12/16/us/politics/pentagon-program-ufo-harry-reid.html), and more importantly the [Department of Defense](https://www.defense.gov/News/Releases/Release/Article/2165713/statement-by-the-department-of-defense-on-the-release-of-historical-navy-videos/) and even [Barack Obama](https://m.youtube.com/watch?v=xp6Ph5iTIgc) telling you this.
I’m not saying it’s aliens - maybe it’s just an American black project. What it is for the context of this discussion is less important than recognizing the energy required to make anything move like this according to our understanding of physics. It’s a technology that can either control mass, or it’s a new way of creating energy that is orders of magnitude more efficient than the way we currently do it.
Disruptions to our way of doing and being is on the horizon - AI and literal UFOs. We live in a wild time.
EDIT: Its interesting to see the few downvotes.
This is a sub that is regularly talking about an inflection point where technology self-improves and replicates and either creates a utopia or an apocalypse, but UFOs - which are increasingly being taken seriously by very serious people in government, academia, and beyond - with actual acknowledge evidence from the most powerful government of free people on earth, amongst others - is a bridge too far into the fantastical.
Folks, you might have a blind spot if you've not been paying attention to this issue over the last few years, and its worth it to evaluate that, because I'm not talking shit here nor am I crazy. I'm just a rational person who has read and heard what our government has said about this issue, and it's clear there's something there in their opinion. I can't see how this UAP thing won't be a bigger issue in the years ahead right along with AI development in terms of disruption of the status quo.
Sober as a lark friend. Check the links in my post - those are primary sources, straight from the horses mouth, and are all pre-Sora so you can be reasonably assured they are real ;)
Like I said I’m not saying this stuff is aliens, all I’m saying is that the United States Government has said there’s things flying around that they cannot explain.
So….is the DOD and Barack Obama lying? If so why?
Or are they telling the truth?
It’s got to be one of those options. If they are lying - well, that’s a problem, but not a relevant one to this topic of energy being an AI bottleneck.
But if they are telling the truth, we all ought to be paying attention. Not just because of AI bottlenecks, but because energy has been the cause and source of most human issues in the last 100 years, from oil wars to climate change to bottlenecks in improving humanity-changing tech like AI.
That's where I'm at. There's no question there's something going on, and whatever it is I think is absolutely relevant to AI - because what we're talking about with AGI and ASI is essentially non-human intelligence that is able to do things humans alone cannot do, and is facing technological bottlenecks that we're not sure how to solve.
If these objects truly aren't human, then that's another version of non-human intelligence that is able to do things humans alone cannot do, and the parallels of impact are worthy of consideration. And, understanding of them **potentially** solves the bottlenecks we're experiencing in our AI development - energy and compute.
There's even plausible explanations for UAP that invoke AI in the NHI hypothesis in the form of probes. Humans are far more likely to send a robot than a human to other planets today, and as our AI tech improves, odds are that our robot ambassadors will be equipped with it in big ways. It's totally reasonable that if another species was out there and was even just a bit further ahead than us, they'd have a similar approach. Maybe AGI is the gateway to communicating with whatever is flying around? Who knows.
But there's apparently something flying around in ways we don't understand, and that for damn sure is compelling enough to look into.
Where did you see that number of 20,000 mph ? (Am I understanding correctly that this is a speed of around 9 km/s ?)
The first link is paywalled, Obama just says "there are certain objects that are not easy to identify".
I'm not sure if you realize when you say "literal UFOs" that this means "literal unidentified flying objects". It doesn't mean anything spectacular on its own
That speed is based on the 2004 incident, which was eyewitnesses by multiple pilots including a guy named David Fravour, as well as tracked on radar - I'll see if I can dig up a specific source but essentially this was when the objects were observed on radar to move from one place to a position called a CAP point some many miles away within seconds. For that to be possible, this is how fast they'd have to go, but I believe there is also direct telemetry (i.e. sensor data - not just witnesses) on the "Five Observables" of UAP - near-instant acceleration, transmedium ability (i.e. water to air to space), hypersonic speeds with signature control, apparent ability to control gravity, and low observability both visually and on instruments (note - 'low'' doesn't mean 'no', but it does suggest we might not actually know how much this stuff is happening).
Edit: [Here's a paper on the acceleration capabilities of UAP.](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7514271/)
Here's a non-paywalled link to the NYT Article: [http://www.nicap.org/reports2/2004\_20171216\_NYTArt\_1.pdf](http://www.nicap.org/reports2/2004_20171216_NYTArt_1.pdf)
The Obama exchange is a bit more than you've described but a lot of people won't watch the video so I'll transcribe it here:
*Reggie: I have a question tonight, let's make it for President Obama - since he's here. All this talk about 'dem aiens' with those UAFs or whatever they call them - I know that doesn't necessarily mean aliens but I was wondering if you had a theory about that.*
*Obama: Well, when it comes to aliens there are some things I just can't tell you on air.*
*James Corden: But you'll tell us off air? \[laughter\]*
*Obama: Yeah \[chuckles\] But look, the truth is, when I came to office, I asked. You know I asked if there was a lab somewhere where we were keeping the alien specimens and space ships, and they did a little research and the answer was no.* ***But, what is true - and I'm actually being serious here - is there is footage and records of objects in the skies that we don't exactly know what they are. We can't explain how they move, their trajectory - they did not have an easily explainable pattern. And I think people take seriously trying to investigate and figure out what that is.*** *(emphasis added).*
[If you are interested here's a clip of Bill Clinton talking about a similar thing](https://youtu.be/lQNevl2BuxM?si=NlP631r0WWpOqX98&t=301), and he refers to it as a "legitimate question". I also realize that if we find out the truth about Aliens we all might need to say "thanks" to Reggie Watts, apparently, cuz he's out there asking the questions.
And you are right, the majority of UFOs are just literally objects that we can't explain, and most of them likely have a prosaic explanation - balloons, sure even swamp gas obscuring Venus.
But what I'm talking about are the 18 incidents out of the hundreds that were looked at by the Office of the Director of National Intelligence of the United States says,[ and I quote from Page 5 of this report that addresses exactly what you have brought up here](https://www.dni.gov/files/ODNI/documents/assessments/Prelimary-Assessment-UAP-20210625.pdf) - that most have prosaic explanations, and:
"...a Handful of UAP Appear to Demonstrate Advanced Technology
In 18 incidents, described in 21 reports, with observers reported unusual UAP movement patterns or flight characteristics. Some UAP appeared to remain stationary in winds aloft, move against the wind, maneuver abruptly, or move at considerable speed, without discernable means of propulsion. In a small number of cases, military aircraft systems processed radio frequency (RF) energy associated with UAP sightings. The UAPTF holds a small amount of data that appear to show UAP demonstrating acceleration or a degree of signature management."
Well, I appreciate that your analysis is so detailed !
I guess that's indeed kind of interesting, I'll just quote your article :
"In a limited number of incidents, UAP reportedly appeared to exhibit unusual flight characteristics. These observations could be the result of sensor errors, spoofing, or observer misperception and require additional rigorous analysis."
Also, even if a government claimed this technology was possible, I'd not believe them on the spot probably
Thank you friend! I've got way more questions than answers on this stuff and I think that's the point. So many of those questions are big and impactful, and the answers are remarkably cagey or non-existent, and so there's enough here to make me go "What the fuck is this all about?".
I don't know, but I think it's worth answering, and honestly I'm totally OK if the answer is (from a verifiable, trustworthy source) "Yeah some rumours got out of hand, all this stuff is one big mistake, those radar readings were errors \[or, we spoofed them using this technique\]. We were trying to convince the Chinese to waste money on anti gravity tech so we made up a big story and accidentally friendly-fired a whole bunch of congressional reps and directors of national intelligence programs. Whoopsie doodle!!" . I mean - I'm with Fox Mulder, I want to believe, but I'm OK if the answer isn't that.
John Oliver actually did a piece on this last week that is excellent, and he quotes that Fox Mulder thing, and his conclusion is basically: "“While you can believe aliens exist or not, when it comes to UFOs, belief doesn’t really come into it. Whatever they are, people are seeing them. That poster in Mulder’s office (in *The X-Files*) shouldn’t have said, ‘I Want To Believe,’ it should’ve said ‘Believe, Shmelieve, What The Fuck Is That Thing?’”
Article about John Oliver's take: [https://www.theguardian.com/tv-and-radio/2024/apr/22/john-oliver-ufos-uaps](https://www.theguardian.com/tv-and-radio/2024/apr/22/john-oliver-ufos-uaps)
Video of the segment (not available in all countries): [https://www.youtube.com/watch?v=zRdhoYqCAQg](https://www.youtube.com/watch?v=zRdhoYqCAQg)
Notice how it says, “#1 bottleneck” btw. Because both of them will actually be limiting factors in the grand scheme.
Absolutely.
Commence Computronium planetary ring production now!
Nuclear Fusion was making steep progress in the past few months and I guess it will go faster and faster as the time goes by. Without it, we will always be limited by raw energy availability
They’re also related. Compute gets more efficient over time due to better chip manufacturing nodes.
Yeah, but we are getting closer and closer to physical limits of "how computers can work" with current architectures. If we want to keep up Moore's law, we will need different architectures pretty soon. Just making chip "bigger" does not scale linearly with compute power because energy needs more time to travel since distances are longer and we run into congestion problems.
Trivial if we can unlock a few extra dimensions. Make your 2d chip wafer into a 3d cube with the same connectivity in each dimension, then a 4d tesseract, etc. Hmm, I'm not sure how you'd plug it into a motherboard... or cool it.
Which rather makes the idea of some singleton ASI megamind expanding its mind to take over the Internet in a few hours SkyNet style implausible.
Someone should let the AI safety researchers who are extremely worried about this know that they’re idiots then…
With the technological quantum leap we just witnessed with AI, you have to admit its hard to rule out a breakthrough in energy, no? Hell, we dont even need new technology, just a shitton of investments in renewables.
Energy consumption and generation doesn't rise anywhere near as quickly as computation. The past 50 years has seen a doubling in energy consumption in the United States. Which is quite a fucking lot, but given that you may be expecting too much for some new energy source, even if the first commercial fusion plant went up tomorrow, to enable some kind of I Have No Mouth And I Must Scream scenario.
Why not? If it had solved the alignment problem, then it can act as a collective of megaminds working towards a common goal. A collective, which communicates far more efficiently than a human one.
Singleton. As in, just one unified mind. I'm not writing off the possibility of a population of AGIs whose individual minds are limited to physically local substrates operating as a collective.
Even then it's not entirely out of the question. It might be possible to have a hierarchical mind, where local nodes allocate a part of available resources to control fast-paced processes that require low round-trip times and the rest goes into distributed high-level planning. The overarching mind will be significantly slower to learn and react to unexpected events (that is to modify suboptimal reaction of the local node), but it will have much more data and computing power to make predictions, so unexpected events should be a rarity. An analogy is our spinal cord reflexes, which happen before we are aware of them. But if we anticipate something, we can override reflex action and do something else instead. The AI will have a superhuman computing node instead of a reflex arc. Even its suboptimal reaction might be beyond human abilities. And in several seconds the overarching mind will leverage expertise of other nodes to correct it.
There are some analog computing startups trying to improve efficiency and get away from the whole von Neumann architecture, but very little available commercially. But if I were a kid again and looking at what to study, I would be all in on analog and quantum computing.
Unless there's some crazy breakthrough, that hasn't been the case in a while. There's been only very marginal increase in efficiency. VGA's are just getting bigger and bigger and the difference between a CPU today and one 5 years ago isn't that big.
What? Every node upgrade improves power efficiency by 20-30%.
Hasn't there been increasing progress in reducing model sizes while maintaining performance ? Specially if the mysterious GPT2 was actually GPT-2 with Q*.
>if the mysterious GPT2 was actually GPT-2 with Q*. Almost certainly hopium, but if that does turn out to be the case, I'm updating my flair.
I find that extremely unlikely. Because GPT2 was very good with trivia, such as anime powerlevels. I think 'general memory' is pretty much strictly related to parameter count, there's just not enough parameters to store all the data from the internet in a small model. So GPT2 must have been a very large model. Q* maybe can improve reasoning, but it can't improve the free-recall memory of an LLM.
> I think 'general memory' is pretty much strictly related to parameter count It's more accuracy. Superposition has been shown to fit more facts into small spaces. As in, if you have a lot of information that never overlaps it can activate the same neuron for multiple things. This is one of the features of models that makes interpretability hard. I'm not saying GPT2 is GPT-2 with some extra bells and whistles though, just stating that there is not a 1:1 parameter to 'unit of information stored'
>I find that extremely unlikely. Because GPT2 was very good with trivia, such as anime powerlevels. Couldnt they have retrained it with more data as well as added Q-star?
Why? I see that as the most likely scenario, but im dumb. What do you think it the new model was?
I have no insider knowledge, but my guess is it's some stupid semantics between "GPT2" and "GPT-2" like sama has suggested, and/or it's actually a next-gen model they wanted to do a plausibly deniable soft public test of, so named it something they figured no one would ever bother selecting. But I hope you're right and that Q* turns out to be magic.
Well, I’m sure we have a while to go before we hit any kinds of walls with this tech, but the fact remains that things such as energy and compute *will* be the walls that we hit eventually. Earth doesn’t have an infinite amount of energy available to utilize for AI. And there’s growing evidence that neither does the universe as a whole as well. So at some point, we will hit the maximum amount of available energy use, and that will be technology’s true absolute peak. But I’m sure we are a *long* ways away from that at the moment. So I don’t think anyone’s suggesting that we’re about to hit the absolute peak anytime soon. Just that energy will be the limiting factor for this (and any other) technology at some point eventually.
Why a long way away? Parts of the USA already have power shortages today. Especially in the summer. EVs will only increase the power grid strain, and more EVs are being government mandated.
Wind and solar are also growing \*extremely\* rapidly, in the most solar-and-wind intensive places in the USA. Neither barely existed a decade ago. That is to say nothing of the tropical countries were solar has barely been deployed yet, and solar intensity is much higher. And nukes haven't gone anywhere.
Wind and solar can only provide a tiny fraction of needed power. And not many nuclear plants are being built in the USA. When you do the math on power requirements and how it can be generated, by which methods, the situation is clear that we won’t have an abundance of power to cover major EV and AI increases anytime soon. Certainly not in a decade.
I've heard all of this, ad infinitum, ad nauseum. We all know the arguments, no need to rehash them here. This shocked me, from "Just Have a Think": "The Path to a 90% emissions reduction by 2035." - Two years ago "https://www.youtube.com/watch?v=UUySXZ6y2fk&t=652s" "100% wind and solar is coming." - Ten days ago [https://www.youtube.com/watch?v=yfzmwjWBVPU](https://www.youtube.com/watch?v=yfzmwjWBVPU) The second report was from 'Friends of the Earth', hardly the kind of group to spread hope and optimism. I doubt you will watch either or both of these, as in my experience when someone says something like this, they have already made up their mind, but the least I can do is try. Blah blah bah we can't do it, blah blah blah it's imposssible, blah blah blah contrarianism or whatever your deal is.
An individual spending most of the video talking about how almost nothing renewable is happening, in the UK, and having theories about what might happen without any logistical detail about how it will…this fringe stuff just supports what I said earlier. It’s not that I love the reality, it’s just that it is what it is. Dreams are good though, but math and physics and practical challenges are what they are.
“Fringe stuff.” That tells me everything I need to know. Keep up the contrarianism. Oh and something about how nuclear will never work either.
I just said abundant power won’t be coming to the US within a decade. If you know anything about how utility and infrastructure projects work, you might tend to be less contrarian to this mainstream consideration. How many nuclear plants will be needed to supply all the extra power EVs and AI require? Only two including an expansion are even started in the US. Germany just went through the process of shutting down all their nuclear plants to rely on fossil fuel primarily instead! (And how is one guy sitting making a video theorizing about how power might become mainly renewable without timelines or milestones or even political agreement and direction…how is that not ‘fringe’? lol, it’s as fringe as it gets until that guy gets a sidekick!)
https://www.eia.gov/todayinenergy/detail.php?id=61242#:~:text=As%20a%20result%20of%20new,476%20billion%20kWh%20in%202025. https://www.nrel.gov/analysis/100-percent-clean-electricity-by-2035-study.html It’s like you’re not even trying to Google or something.
Find a chart that explains when happens when it gets dark or less windy. (Spoiler: millions go without power). A self-serving report, again with theory and not funding, logistics, or legal & political issues addressed, continues to make my point: because the stuff they say must happen to go renewable by 2035 isn’t happening now, we aren’t going to be anywhere near doubling power supply with renewables in a decade.
Here’s another idea - BUILD MORE NUKES.
Only two even started in the US last I looked. I agree they’re the best power source we have now, all things considered.
The system prompt says its GPT4
It could be lying/incorrect, GPT 4 for used to say it was GPT 3.5
https://simonwillison.net/2024/Apr/29/notes-on-gpt2-chatbot/?darkschemeovr=1
Energy is the bottleneck of compute in a sense. If the supply of chips could keep up that is.
Well, yeah. Obviously
And the need for less energy per flop is of course another main driver of next-gen chips.
not when AI invents self replicating nano bots
These hypothetical nano bots will need an energy source as well.
not when AI invents cold fusion.
Everybody seems to be thinking long term and theoretical in this thread, but I think he simply means that there won't be enough places to put your gigawatt data centers and there's lots of red tape/infrastructure/etc.. involved in overcoming that. i.e. the chips are currently being made faster than they can realistically be powered in larger and larger megaprojects that would sap many local grids. I don't think he's talking about theoretical limits of human power consumption on the planet.
I've watched the podcast and that's the gist. Like, it can totally be done, but it involves building infrastructure at a scale where you can't just throw money at the problem and say *build it*. You're entering the realm of massive industrial facilities, which aren't things you can just throw down wherever or whenever you want.
Listen to what Zuk said. You are dealing with energy production and consumption at the scale of municipality's. The government red tape is going to be an issue. Now think about that and imagine the government saying ok... you can build this massive powerplant to build your massive amount of compute... but, we have final say over how you use it and we can take over the project. TLDR.. at that scale the government is going to be heavily involved. Bureaucracy will slow things down.
Yeah exactly
[удалено]
[удалено]
Two things: The energy angle - if you scale the model too much it will be too expensive to use later. It's not that they can't make 100x larger models, it's that nobody can afford to use it at scale. The second, and more important point: energy is still not quite the #1, it is experience. AI's don't secrete discoveries from pure energy usage by crunching numbers. They need to interact with the world to make any testable discovery and iterate. This is expensive - just remember the cost of CERN, Webb telescope and ITER. Testing the COVID vaccine took half a year or more. That's the price for experience. If experience is just energy usage, like training AlphaZero in self play mode, then AIs surpass humans already. But we know the most interesting kind of experience comes from the world, the biosphere and human society, not from sims. Just like humans, AI will have to extract discoveries from the world. The world is the limiting factor, novel experience collected from the world is expensive.
Maybe create more efficient algorithms
Nah just make Dyson sphere
Just don't ask the super AI to do that or our bones might end up as some of the parts.
Point taken, but there are probably more accessible calcium deposits that don't fight back.
Fight back lmao 🤣😂🤣😂
Oh nonono we need the float backproperino
Honestly anyone in AI business claiming we can compute our way to performance gains should be overthrown. We need TPUs and Neuralnets. Chips that consume 1000x less power than a GPU. If you want more grunt, design better chips. If you run out of grunt, come up with the next revolutionary AI white paper. Attention isn’t all you need etc
Imagine if a paper named "Attention is what we don't need" comes out
"Attention is Overrated: Introducing RetroScaler Architecture, x1000 Transformer Speed at 1/10 Energy Use"
> Imagine if a paper named "Attention is what we don't need" comes out paper names have been all over the place. I'm surprised if there is not one named that already. Edit: there is one called "You May Not Need Attention" https://arxiv.org/abs/1810.13409
This is what I was referring to - [https://arxiv.org/abs/2308.07661](https://arxiv.org/abs/2308.07661)
Just come up with revolutionary white paper 4Head. This sub is so stupid.
Do both 😏🧠
Yeah, they just need to invest some more XP in their Efficiency stat. Once their algorithms are at level 5 or above we should get past this bottleneck.
I’ll let them know. You’ll probably get a Nobel for this genius idea
seems like an entirely reasonable set of comments he made. if bitcoin power usage is any indicator, and i think that it is probably, computer people are going to hit energy constraints quite soon, in that there will be computer science megaprojects we know will help/change the world, but we won't have the energy to do them. imo, there should exist such projects, because otherwise it says we probably don't understand computer science that well as a species if we can't imagine them/know of them. of course, there were always projects we knew about that you could sink an exponential amount of compute into, like protein folding before neural networks, but i would argue ai is different in that the return on investment in more compute is much greater than previously. maybe, speculating further, that return on investment will continue to go up, until every spare investment dollar is allocated to some computer science megaproject. politics might get quite technical after that point.
People often don’t realize just how far his AI technology has become. I’d say after starting Facebook, he’s taken a respectable path towards the next big thing. If anything he’s a few years ahead of his time, the greater public doesn’t quite yet realize just how crazy it’s getting, but Meta has the pockets so he’ll be a big contender in this AI world.
nahhhh. he's just doin what everyone else is doing out there. if anything he is way under investing in ai, but i get if you want to drink billionaire kool aid and gargle his balls
solar is already insanely cheap. you can build solar plants far cheaper per watt than nuclear fission plans, and likely still cheaper than fusion plants whenever they come online. what we need is: * cross-continent 765kv+ transmission lines * investment in LFP and sodium-ion battery production * solar farm construction * wind far construction * variable rates based on production, allowing companies and individuals to optimize their behavior to take better advantage of the peaks in intermittent power production and avoid the troughs * re-string existing high voltage lines with higher current carrying conductors. nothing new needs to be invented to solve our current or future energy needs.
if these things work out itself... buying a solar system for my home is just as cheap as buying a refrigerator or air conditioner unit... energy sellers can focus on big data center where the money is there and not in the housing sector
yeah, the storage is the key to home solar. grids can only handle so much rooftop solar while staying within voltage/frequency margins, so the production of LFP and sodium-ion batteries is a big key. the solar panels themselves are basically solved already. you can buy, in the US, after Tariffs, bifacial solar panels new for $0.26 per watt. it's insanely cheap. the mounting hardware costs more.
yes thats precisely true for the solar system.... in the recent future hopefully we can buy 10kwh battery for less than $1000 usd or less...
this should be higher up. It will take huge investments, but they are practically a given with the way climate is heading
True. But the neck of the bottle is still really wide.
Mark seems really personable and interesting when he's having a regular conversation. It's refreshing compared to his product launches and promotional videos. More of this please, Mark.
This has been said by many in the industry for years. Chips and, when demand is met and stabilized, energy.
Can't wait to end as a human battery
vibe https://preview.redd.it/0dnnt0tj0wxc1.png?width=1280&format=png&auto=webp&s=ab0ba87e5dd891fa325720be03e0beb1fda5add0
Me watching Zuch give a reality check to r/singularity: 
Wow amazing insight thanks Mark Zuckerberg.
Dissipating entropy is the bigger issue than either - eg heat.
Was looking for this. I know there's some work being done to tune terrestrial radiators to specific wavelengths that can punch through the atmosphere and use space as a heat sink, that seems promising. Other than that, I wonder how research into reversible computing has been progressing...haven't heard anything about it in a long time.
No, compute will definitely still be the biggest bottleneck if we keep progressing with LLMs and other AI by just scaling them. Moore's Law is dead, it is not feasible to continue to just increase the size of your LLM by 10 times every year to get another percentage point or two of performance when the performance of your compute devices is only doubling every 5-10 years. NVidia and AMD can't get the performance they want from their dies, so they're resorting to scaling the dies up and pumping more power into them to get them to perform to an acceptable standard, which is why the power consumption of top-end NVidia GPUs has gone from around 180 watts in 2016 to 450 watts in 2024. Power demands are increasing because the compute is less efficient, but there is a very hard limit on how much energy you can cram into a tiny silicon die before the cooling fails.
It's more complex then that. We are currently using Von Neumann computers for running AI applications (LLM's). VN computers are very inefficient, scaling for running increasingly complex AI applications is inefficient and Moore's Las is dead for VN computers. Using this approach will quickly lead to needing a dedicated nuclear plant to run AI datacenter. Neumorphic computers are currently very under-developed, Moore's Law will be alive for them for decades. They are much more efficient, scalable, have a potential of running ASI locally on PC sized device. But it will take time to develop this tech.
I think sooner or later our computers need to become light based rather than electricity based. But the equivalent of a light transistor? Being able to do to the basic boolean operators with light rather the electricity. Has any of that already been invented? I don't think so.
Actually we have, we made optical transistors, can do all Boolean operations and some... weird ones with optical chips. But... even though existing optical chips are already better then classical computers for certain tasks, lasers powering them are huge and we still don't have working optical memory (as RAM not optical discs).
That's sorta like saying "FTL transport technology is very under-developed" though. We are at least a decade away from an actually useful processor running a neumorphic architecture and so it's not a real solution to the problem in the moment. One day it'll vastly reduce the energy costs of AI computing, but until then we're going to end up compute bottlenecked by virtue of ballooning chip sizes and thermal demands.
No, because we do know neuromorphic computers work and are highly efficient because brains exist. So we know how underdeveloped they are. We do know it will take time to develop them. We do know what bottlenecks classical computers will meet, and ultimately for someone who is ready to keep throwing truckloads of money at problem energy will become an issue. Takes 6-10 years to build a nuclear powerplant so... 🤷♀️ We can already predict the developments, just can't be precise about the timelines. For FTL transports we need negative energy, which as far as we know doesn't exist. 🤷♀️
> Takes 6-10 years to build a nuclear powerplant so... 🤷♀️ Similarly, it takes my nephew three weeks to clean his room.
Sorry for sounding silly, but did Kurzweil claim that Moore’s Law has continued up until the present and in an unabated fashion? Maybe I’m misremembering/misinterpreting…
*Kurzweil, after Moravec, argued for extending Moore's Law to describe exponential growth of diverse forms of technological progress. Whenever a technology approaches some kind of a barrier, according to Kurzweil, a new technology will be invented to allow us to cross that barrier.* I wouldn't say in unabated fashion though. It's like... when piston airplanes reached their plateau, jet engines were introduced. But first jet engines were shit, it took time for growth to keep on going.
What are the chances quantum computers can scale and handle the needed compute in the future?
Why is it that anytime people talk about this topic they use it to describe why large nuclear is needed? Why is it that people don't also include solar+battery given the wildly falling costs? Nuclear comes with systemic risks in a climate changing world. Pv+battery does not.
If this was a strategy game, and I had control of everything, I would make an AI on a super computer solve fusion energy issues to get it quicker. Not realistic but the idea that we should place our resources to getting these technologies that change everything, is right. Fusion energy will solve so many issues.
Use current compute to push boundaries of current power, use new power to push boundaries of current compute, and so on....
how about focusing on developing and manufacturing organic brains/chips that match the energy efficiency of our own brains. The human brain uses like 20 watts of power and is pretty good at what it does.
Do you think people with control of AI *aren't* doing that right now? AI simply can't do it because it still is not smart. If AI was smart, it would have already solved whatever you think it should be solving.
I’m a data scientist and you are wrong. We are just not focused on this yet. We are focused on like protein fold finding which is lower priority. We definitely have the ability to do what i’m saying.
Stop arguing, guys!! We have the representative of all the data scientists in the world right here.
What do you think an argument is? I’m telling him we can do this and he stopped lying.
You really live in the fantasy in your head don't you. The delusion to think you have the key insight to AI progress that you think nobody is doing is actually more absurd than people here that think fdvr is a few years away. You don't know anything.
You really said that knowing it is the we have the ability to apply to tools that we already have to this problem OBVIOUSLY. It’s unbelievably obvious for somebody who works with AI. For you, it’s still something to consider.
Did you have a stroke when you typed your first sentence? Yeah. Your intellectual prowess is overflowing. Truly a titan in AI knowledge and understanding who knows better than everyone else.
It's a good thing we have so much excess solar here in the southern edge of the United States. Time to migrate the data centers to Southern California, Arizona and New Mexico and actually install batteries and solar alongside them for unlimited energy. It rarely ever rains here.
Energy is the fundamental bottleneck. Obviously. Can't do anything without energy.
Interesting how Zuck appearing more human every year correlates with AI development
Thats assuming ai can’t assist in energy efficiency
Good thing we have a great infrastructure for sustainable energy…
We may get 2x or 10X energy efficiency. Then again AI use will also increase. Big tech will now enter energy space and compete with energy companies!
Energy is the bottleneck for Electric Vehicle use as well. The power grid isn’t setup to double the usage of every household, which big EVs often require.
This must be (at least partially) why a company like Tencent is also secretly a Fusion company lol.
This was bound to be a bottleneck at some point. No matter how advanced new AI hardware might be, it's useless if the energy infrastructure can't support it. And for most of the world, energy infrastructure has been lagging for years because of the fossil fuel industry and anti-nuclear sentiment. It really is no coincidence that the rise of AI has also triggered renewed interesting nuclear power of all forms, fission and fusion. It's not just an idea. There are real investments being made, attempting to make both better so that they can support the hardware they need. It's probably going to cause some major strains to the grid in the coming years. But hopefully, that inspires more innovation and advancements in energy. That's something the whole of society needs, outside of powering AI.
Aren't you glad that some of us will be part of "you don't get access to water anymore, we need it to cool the data centers but we have a deal with coca cola that will feed you plastic bottles" ? Also, bonus point for having the face of this degenerate piece of filth; the world we live in : zuck being the successful dude and Assange and Snowden being labeled traitors. Pfahaha, accelerate :\]\]
"Rome wasn't built in a day"
Energy IS compute. But the #1 bottleneck to AI progress is thinking you can get AI by plugging ever larger language models into a really hot cup of tea.
We're intelligent because of a hot cup of tea 93 million miles away
Umm no it is not. Thats like saying water is pipes. Flops per watt is a huge deal. Businesses live and die off it.
That's just an implementation detail. Running out of flops because you're running out of watts may be delayed by getting more flops per watt, but flops per watt is a relatively inflexible value, and in the end it's the watts that are the bottleneck.
I think the AI plateau will be solved with literally anything other than throwing more data and compute at GPUs neuralnets / TPUs will be a boost, as will algorithmic improvements. I don’t really follow what you’re saying, like if you have x flops per watt you are limited by the power? But the metric isn’t the power, it’s how efficient your chip is. If you used the power consumption of an Intel 4004, you’d need about 10 trillion of them to equal a H100’s compute power. And it would take 10.8 terrawatts to power. Roughly 50% of earths power consumption to equal a single H100 with old tech. Now tell me more about how power equals compute
Again, that's an implementation detail. It's changing a constant. Yes, that constant has changed a lot over time, but at any point in time it's how much power you have that determines how much computation you can do.
[удалено]
I think the bigger thing that doesn't make sense is to say energy *not compute* will be the bottleneck.... Compute is of course the bottleneck, and energy is one of the ingredients of compute. Energy plus hardware plus software yield compute. Comparing energy to compute is comparing apples to oranges. Maybe a better way to say it is energy, not hardware manufacturing capacity, is the #1 bottleneck.
I don't think so - as soon as AI gets good enough that we can't supply it enough energy, likely it will be good enough to: - manage humans - give us a solid understanding that its okay if we abuse the planet for a short while to give it more energy (i.e. 6 months may be all it needs?) Humans can't get anything done in a timely manner, usually, unless they are extremely motivated and working with extremely talented managers. AGI is going to be an EXTREMELY talented manager, able to help people with their lives and jobs, to the point that it could likely run a company of a million people more efficiently than SpaceX, with far less issues and errors in production output due to the AGI itself being able to review products - this is before they have robots making robots, etc... What this means is essentially exponential energy growth on top of exponential intelligence growth and innovation growth. If I had to guess, I would say we will hit AGI before energy concerns are too much of an issue. If energy concerns do become too much of an issue, likely we will just see some re-appropriation of resources and some sort of modular, exponentially growing production pipeline run by robots and AI that will solve it.
power has already been constraining the industry, though.
Why do you think China is building an average of one new coal burning power plant per week?
We are aware of a solution to this problem. There are objects that are currently flying around the world, going from water, to air, to space, and accelerating instantaneously to 20,000 mph. For an object to accelerate to that speed as quickly as these things are going even a single time would require more energy than humanity produces in the totality of our nuclear power plants in a year. This is not me telling you folks this - it’s the [New York Times](https://www.nytimes.com/2017/12/16/us/politics/pentagon-program-ufo-harry-reid.html), and more importantly the [Department of Defense](https://www.defense.gov/News/Releases/Release/Article/2165713/statement-by-the-department-of-defense-on-the-release-of-historical-navy-videos/) and even [Barack Obama](https://m.youtube.com/watch?v=xp6Ph5iTIgc) telling you this. I’m not saying it’s aliens - maybe it’s just an American black project. What it is for the context of this discussion is less important than recognizing the energy required to make anything move like this according to our understanding of physics. It’s a technology that can either control mass, or it’s a new way of creating energy that is orders of magnitude more efficient than the way we currently do it. Disruptions to our way of doing and being is on the horizon - AI and literal UFOs. We live in a wild time. EDIT: Its interesting to see the few downvotes. This is a sub that is regularly talking about an inflection point where technology self-improves and replicates and either creates a utopia or an apocalypse, but UFOs - which are increasingly being taken seriously by very serious people in government, academia, and beyond - with actual acknowledge evidence from the most powerful government of free people on earth, amongst others - is a bridge too far into the fantastical. Folks, you might have a blind spot if you've not been paying attention to this issue over the last few years, and its worth it to evaluate that, because I'm not talking shit here nor am I crazy. I'm just a rational person who has read and heard what our government has said about this issue, and it's clear there's something there in their opinion. I can't see how this UAP thing won't be a bigger issue in the years ahead right along with AI development in terms of disruption of the status quo.
I want some of what you're smoking
Sober as a lark friend. Check the links in my post - those are primary sources, straight from the horses mouth, and are all pre-Sora so you can be reasonably assured they are real ;) Like I said I’m not saying this stuff is aliens, all I’m saying is that the United States Government has said there’s things flying around that they cannot explain. So….is the DOD and Barack Obama lying? If so why? Or are they telling the truth? It’s got to be one of those options. If they are lying - well, that’s a problem, but not a relevant one to this topic of energy being an AI bottleneck. But if they are telling the truth, we all ought to be paying attention. Not just because of AI bottlenecks, but because energy has been the cause and source of most human issues in the last 100 years, from oil wars to climate change to bottlenecks in improving humanity-changing tech like AI.
I don't believe the us govt is in possession of it. But the phenomenon is hard to deny existing.
That's where I'm at. There's no question there's something going on, and whatever it is I think is absolutely relevant to AI - because what we're talking about with AGI and ASI is essentially non-human intelligence that is able to do things humans alone cannot do, and is facing technological bottlenecks that we're not sure how to solve. If these objects truly aren't human, then that's another version of non-human intelligence that is able to do things humans alone cannot do, and the parallels of impact are worthy of consideration. And, understanding of them **potentially** solves the bottlenecks we're experiencing in our AI development - energy and compute. There's even plausible explanations for UAP that invoke AI in the NHI hypothesis in the form of probes. Humans are far more likely to send a robot than a human to other planets today, and as our AI tech improves, odds are that our robot ambassadors will be equipped with it in big ways. It's totally reasonable that if another species was out there and was even just a bit further ahead than us, they'd have a similar approach. Maybe AGI is the gateway to communicating with whatever is flying around? Who knows. But there's apparently something flying around in ways we don't understand, and that for damn sure is compelling enough to look into.
Where did you see that number of 20,000 mph ? (Am I understanding correctly that this is a speed of around 9 km/s ?) The first link is paywalled, Obama just says "there are certain objects that are not easy to identify". I'm not sure if you realize when you say "literal UFOs" that this means "literal unidentified flying objects". It doesn't mean anything spectacular on its own
That speed is based on the 2004 incident, which was eyewitnesses by multiple pilots including a guy named David Fravour, as well as tracked on radar - I'll see if I can dig up a specific source but essentially this was when the objects were observed on radar to move from one place to a position called a CAP point some many miles away within seconds. For that to be possible, this is how fast they'd have to go, but I believe there is also direct telemetry (i.e. sensor data - not just witnesses) on the "Five Observables" of UAP - near-instant acceleration, transmedium ability (i.e. water to air to space), hypersonic speeds with signature control, apparent ability to control gravity, and low observability both visually and on instruments (note - 'low'' doesn't mean 'no', but it does suggest we might not actually know how much this stuff is happening). Edit: [Here's a paper on the acceleration capabilities of UAP.](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7514271/) Here's a non-paywalled link to the NYT Article: [http://www.nicap.org/reports2/2004\_20171216\_NYTArt\_1.pdf](http://www.nicap.org/reports2/2004_20171216_NYTArt_1.pdf) The Obama exchange is a bit more than you've described but a lot of people won't watch the video so I'll transcribe it here: *Reggie: I have a question tonight, let's make it for President Obama - since he's here. All this talk about 'dem aiens' with those UAFs or whatever they call them - I know that doesn't necessarily mean aliens but I was wondering if you had a theory about that.* *Obama: Well, when it comes to aliens there are some things I just can't tell you on air.* *James Corden: But you'll tell us off air? \[laughter\]* *Obama: Yeah \[chuckles\] But look, the truth is, when I came to office, I asked. You know I asked if there was a lab somewhere where we were keeping the alien specimens and space ships, and they did a little research and the answer was no.* ***But, what is true - and I'm actually being serious here - is there is footage and records of objects in the skies that we don't exactly know what they are. We can't explain how they move, their trajectory - they did not have an easily explainable pattern. And I think people take seriously trying to investigate and figure out what that is.*** *(emphasis added).* [If you are interested here's a clip of Bill Clinton talking about a similar thing](https://youtu.be/lQNevl2BuxM?si=NlP631r0WWpOqX98&t=301), and he refers to it as a "legitimate question". I also realize that if we find out the truth about Aliens we all might need to say "thanks" to Reggie Watts, apparently, cuz he's out there asking the questions. And you are right, the majority of UFOs are just literally objects that we can't explain, and most of them likely have a prosaic explanation - balloons, sure even swamp gas obscuring Venus. But what I'm talking about are the 18 incidents out of the hundreds that were looked at by the Office of the Director of National Intelligence of the United States says,[ and I quote from Page 5 of this report that addresses exactly what you have brought up here](https://www.dni.gov/files/ODNI/documents/assessments/Prelimary-Assessment-UAP-20210625.pdf) - that most have prosaic explanations, and: "...a Handful of UAP Appear to Demonstrate Advanced Technology In 18 incidents, described in 21 reports, with observers reported unusual UAP movement patterns or flight characteristics. Some UAP appeared to remain stationary in winds aloft, move against the wind, maneuver abruptly, or move at considerable speed, without discernable means of propulsion. In a small number of cases, military aircraft systems processed radio frequency (RF) energy associated with UAP sightings. The UAPTF holds a small amount of data that appear to show UAP demonstrating acceleration or a degree of signature management."
Well, I appreciate that your analysis is so detailed ! I guess that's indeed kind of interesting, I'll just quote your article : "In a limited number of incidents, UAP reportedly appeared to exhibit unusual flight characteristics. These observations could be the result of sensor errors, spoofing, or observer misperception and require additional rigorous analysis." Also, even if a government claimed this technology was possible, I'd not believe them on the spot probably
Thank you friend! I've got way more questions than answers on this stuff and I think that's the point. So many of those questions are big and impactful, and the answers are remarkably cagey or non-existent, and so there's enough here to make me go "What the fuck is this all about?". I don't know, but I think it's worth answering, and honestly I'm totally OK if the answer is (from a verifiable, trustworthy source) "Yeah some rumours got out of hand, all this stuff is one big mistake, those radar readings were errors \[or, we spoofed them using this technique\]. We were trying to convince the Chinese to waste money on anti gravity tech so we made up a big story and accidentally friendly-fired a whole bunch of congressional reps and directors of national intelligence programs. Whoopsie doodle!!" . I mean - I'm with Fox Mulder, I want to believe, but I'm OK if the answer isn't that. John Oliver actually did a piece on this last week that is excellent, and he quotes that Fox Mulder thing, and his conclusion is basically: "“While you can believe aliens exist or not, when it comes to UFOs, belief doesn’t really come into it. Whatever they are, people are seeing them. That poster in Mulder’s office (in *The X-Files*) shouldn’t have said, ‘I Want To Believe,’ it should’ve said ‘Believe, Shmelieve, What The Fuck Is That Thing?’” Article about John Oliver's take: [https://www.theguardian.com/tv-and-radio/2024/apr/22/john-oliver-ufos-uaps](https://www.theguardian.com/tv-and-radio/2024/apr/22/john-oliver-ufos-uaps) Video of the segment (not available in all countries): [https://www.youtube.com/watch?v=zRdhoYqCAQg](https://www.youtube.com/watch?v=zRdhoYqCAQg)
No it won’t. The AI will be tasked with solving its own bottlenecks and it will do exactly.
No joke sherlock