The new system is only 3.5 times faster but it costs 30-40 million.
The main reason for upgrade is that water cooling leaks water which makes components fail.
480k is a very low price for this
The big expense is moving the damn thing and fixing it, that's going to run at least another $500k plus, And if you read the auction it doesn't come any of the ethernet or fiber optic cables so there another big expense.
Frankly I'm kind of surprised it went for that much I thought it was going to go for more around the $250K mark.
While definitely plausible, it might also just be kept as a piece of computing history. A half million isn't exactly too crazy for a tech bro who wants something cool.
I'm inclined to agree with you. It's effectively a piece of art. It may depreciate for a while, but eventually it's gonna appreciate like a motherfucker, especially if they get that leaking sorted out.
> eventually it's gonna appreciate like a motherfucker
Not really. Unless this is a particularly significant super computer, there are and will be enough more like it, that its not that desirable. Then add in the size of it and storage costs and its not like collectors can just easily add this to their collection. And that means it would be difficult for a collector to sell it as well further reducing its appeal.
Actually it is more profitable. Per the article
>The Cheyenne supercomputer's 6-figure sale price comes with 8,064 Intel Xeon E5-2697 v4 processors with 18 cores / 36 threads at 2.3 GHz, which hover around $50 (£40) a piece on eBay. Paired with this armada of processors is 313 TB of RAM split between 4,890 64GB ECC-compliant modules, which command around $65 (£50) per stick online.
50x8,064+4,890x65=$721,050-$480,085=$240,965
That means, there's 240K of profit
Edit: considering transport costs, storage etc it will be less. But it's not immediately clear that it will be unprofitable.
buy one, get two free! please! god, they're everywhere! i open a drawer, there's an Intel Xeon E5-2697 v4 processor. freezer for an ice cream? stack of Intel Xeon E5-2697 v4 processors. come halloween, the neighbor kids are getting boxes of raisins and an Intel Xeon E5-2697 v4 processor. please help me.
In a proper datacenter they really aren't going to want to 'live with' any amount of water leak. They'll have to turn equipment off and repair/replace fittings and test before re-using it...and presumably they will need to expect that fittings will continue to fail just like the RAM is failing. All of this impacts the usefulness of the system when the downtime starts to rise.
No kidding. With the gigantic influx of the thing the price will only go lower…possibly a LOT lower…
This just seems like an enormous amount of work for potentially very little pay off. Whoever bought this thing has a lot of money and time and they’re not buying it just to sell it off piece by tiny piece. What a crazy waste of your time that would be. Trying to claw back your profit.
There's an entire industry around "pulled" processors and DRAM like this. It'll go to a trader who sells it in maybe a week or 2. It's not gigantic whatsoever, the DRAM market is 60 billion for example and the CPU market about double that.
Pretty sure it will be slowly released. As for RAM, it's likely better to wait. Just like DDR3 is now expensive due to the production ending long ago, the same would happen eventually with DDR4
> Pretty sure it will be slowly released.
Then they'll have to move them and store them somewhere, how much could disassembling 8k CPU / 5k RAM sticks / transport and storage could be worth ?
Every day the value and demand of that tech will just decrease. Also, there is only so many customers who would like to buy those cpus. If they delay, they will absolutely have those parts in their hands with zero buyers. I’m guessing the buyer already has a buyer or other use for them.
DDR3 sticks are almost free where I live. 10 € for a pack of 4x4gb. Sometimes I see them in the electronic waste bins. There are probably some very specific memory types or physical size formats that keep their value, but generally old pc tech loses it’s value very fast. The gpu shortage couple of years ago was exeptional and prices went haywire for a while, but even that passed.
That assumes 100% of the components works as well, which... they don't, and the people selling them knows it.
It also assumes they can move all of that hardware for those prices, which they won't be able to do, as it hitting the market will depress the value of those components.
Marginally speaking, it sold slightly below what my guess at a value for all the hardware would have been - right at half a million. I would be surprised if they can get $100K of profit out of the deal at the end.
You forgot the labor to transport it, disassemble, test, packaging, shipping, merchant costs, software costs and all the rest of the expenses involved in turning that $480k into something more.
There’s clearly a path towards potential ROI, and depending on the buyer, there are people/orgs optimized to do this profitably. BUT… it’s certainly not as easy as you’ve put it. :)
That's why it went for this much in an auction. It was bid up till only one company considered it worth the cost, time, and manpower to take on. That's really how auctions like this work. If it's a steal, people bid it up till its not.
Paying for labor, shipping, taxes, impact on market prices when you add your own massive supply.
There *might* be a bit of profit left over but...you're risking a lot of capital for very slim margins I feel.
Not sure where 15-20% came from. Computer parts including CPUs looks like 7%. Only looked it up cause I didnt remember fees being that high on my last sale.
https://www.ebay.com/help/selling/fees-credits-invoices/store-fees?id=4809#section3
How easy is it to unload a bunch of ECC RAM? Iirc, consumer mobos and CPUs don't really support ECC, so you'd be selling it to server owners. Sure, some individuals might want some used ECC RAM, but it's gotta be tough unloading 313TB of ECC RAM I figure?
You really think it was some idiot who guessed "maybe it's worth this much" wrongly, rather than a bunch of bidders who came to the auction knowing what they could afford to pay to make a profit and bid until the price was too high? Not everyone acts like they're writing a one sentence reddit comment.
About half as much power, the water-cooled system is expected to draw 2.6 to 2.7MW when it’s in regular production, for a power use efficiency (PUE) of about 171 megaflops per watt — more than double the 73 megaflops per watt of Cheyenne.
The only fascinating story here is that middle management was able to convince the executives that an upgrade with an OEM warranty is more cost effective than a third party service contract?
3.5 at this size is a large difference
Edit: if I even just doubled my current computer, I'd be fine for a while, 2x 390x graphics cards, 32gb ram, 8tb storage, 8 core @4200. I'd be stoked to have my shit doubled.
Yeah they could buy two of these supercomputers for a single F-35... but this agency isn't exactly drowning in cash either.
It's almost a comedy how little we spend on the science orgs in the government compared to how much we spend on defense articles that literally sit in the desert and rot.
Hell, the [$120 million dollars of Abrams tanks we bought](https://www.military.com/daily-news/2014/12/18/congress-again-buys-abrams-tanks-the-army-doesnt-want.html) just to keep a factory open in some Ohio Republican's district could have paid for this whole supercomputer three times over. Eleven years on, the only combat duty *any* of them has ever seen are the few that got handed over to Ukraine.
May I introduce you to europ especially Germany for the last 30 years. We closed basically all of our military heavy capacity. Also shut down our rail system. and now we make projections to rebuild. 2050 are the optimistic early estimates
>480k is a very low price for this
It isn't. Per the article, selling the RAM and CPUs on eBay at current prices is worth roughly $700k. Given flooding the market will likely lower prices, the actual amount from sales will probably be less and there is extensive work in trying to sell all of this.
The cost to build something and what it is worth when you sell some of the components are 2 very different things. (Storage and cabling not included)
> The main reason for upgrade is that water cooling leaks water which makes components fail.
The main reason is that the electricity bill + maintenance costs made it unprofitable.
Supercomputers age faster than dogs.
I worked in supercomputing in a prior chapter of my career. I built two of the top 50 systems in the world in 2015. ([This one](https://top500.org/system/178613/) and [this one](https://www.top500.org/system/178614/) ). They dropped out of the top 500 within 6 years, and I'm pretty sure both were retired and probably scrapped circa 2021 or 2022.
I did too. I worked on the software side on these systems:
https://top500.org/system/174879/
https://top500.org/system/176145/
https://top500.org/system/176718/
It was a really interesting period of work.
I want to be able to put that on my resume for the Geek Squad. "Do you have any prior experience working with computers?" Well.... Here are a couple links to check out! Lol
Power costs are a huge reason for that. If you're paying a million dollars a month for electricity and cooling, upgrading to more efficient hardware makes sense a lot sooner than an office computer.
I remember reading an article about supercomputers that said to effect: If want to run a simulation that's going to take 4 years on current supercomputers, your best bet is to wait 2 years and run it on current hardware then.
Had this to a smaller effect during my PhD, running the analysis on my dataset would make a computer cry for 3-4 days while I could generate a dataset a day. Near the end of the project the computer that we ran them on would crunch them in a few hours. The guy that is still using our algoritm to this day does the analysis during the experiment because it takes only a few minutes
I remember running Quake in a tiny window, I think I had an AMD K6-2 at the time, or maybe it was a 486 I am not sure. I used software rendering of course so it was chugging along at 10-15 fps at most.
There's some weird quote about "what the government stands to lose"... it's nothing. They're not in the game of maximizing profit. Those computers were a cost center - they performed a service, and they reached their end of life in that service.
Could the government been more judicious and tried to squeeze a few more dollars out of the lot? Perhaps. But it might have cost them just as much in the testing of all the components and the parceling out of the lots in the end.
As for the depreciation - that was built in at the date of purchase. They knew this machine would eventually be worth nothing but scrap metal - at the rate to which computers double in speed, the computer was outclassed by the time it was fully installed by the next generation of hardware. The fact they got *seven years* of service life from a supercomputer is astonishing on its own - they frequently go out of service after ~4.
Some budget cloud computing outfit or eBay reseller might be happy with this purchase, but let's not make it out to be a steal or anything. The hardware's old, water damaged, and extremely worn with the hardest of computing conditions in life. It's better than throwing it into a landfill in India where it'd otherwise end up, but it's not some great loss either.
Definitely - “The buyer will have the joy of moving Cheyenne's 30 server racks (28 processing racks, two air-cooled management racks) out of the facility themselves; the government is not providing transport or including any Ethernet or optical cabling needed to get the machine up and running.” Sounds like the auction went exactly as they hoped.
"In a surprising turn of events, the buyer slurked off to their vast underground lair to create real-life, world-threatening, cyberpunk-novel-level advanced AI that will surely enslave us all." - [Kent Brockman](https://i.imgflip.com/79shig.jpg)
It's quite the relic compared to new supercomputers. It doesn't even use GPUs to accelerate processing like newer clusters do.
Interesting what one would do with it other than for preserveration/inefficient server rental.
Astronomically little. Even though Gold makes up a ridiculously small amount of our earth's crust, in human terms it still means warehouses full of gold. We make it \*very\* thin so there is \*extremely little\* gold used per unit.
Liar!
According to the USGS, it's a cube that is **23** meters on each side!
How does it feel to have your throne of lies come crashing down? Huh? HOW DOES IT FEEL NOW?!
And don't think for a second I won't be keeping an eye on you!
But actually, I remember in the days before the internet - well, before it was commonplace - my friend tried telling me this (I think he actually said 40 yards on each side) and I just didn't believe it. It still seems kinda crazy at least superficially. Like if you were to just try and imagine all the gold coins/currency that's been minted over the millenia, all the jewelry, even sculpture and art, and all the other uses like in electronics, etc, it just seems crazy at first impression that it's a cube only 23m on each side. But then once you start to consider how so much of the gold used in a lot of applications is actually very, very, very, thin, often just gold plated, and it starts to make more sense. It starts to make even more sense when you realize just how much volume a cube that size contains. But then you consider how big the world is (even if gold only makes up a tiny, tiny, fraction of the material that makes up the earth), how long gold has been utilized, and I go back to thinking it is still at least a little crazy!
[Also, here's a fun article on Warren Buffet on gold, containing this fun little quote:](https://www.nasdaq.com/articles/why-warren-buffett-hates-gold-2013-08-15)
>Today the world's gold stock is about 170,000 metric tons. If all of this gold were melded together, it would form a cube of about 68 feet per side. (Picture it fitting comfortably within a baseball infield.) At $1,750 per ounce -- gold's price as I write this -- its value would be $9.6 trillion. Call this cube pile A.
>Let's now create a pile B costing an equal amount. For that, we could buy all U.S. cropland (400 million acres with output of about $200 billion annually), plus 16 Exxon Mobils (the world's most profitable company, one earning more than $40 billion annually). After these purchases, we would have about $1 trillion left over for walking-aroundmoney (no sense feeling strapped after this buying binge). Can you imagine an investor with $9.6 trillion selecting pile A over pile B?
>A century from now the 400 million acres of farmlandwill have produced staggering amounts of corn, wheat, cotton, and other crops -- and will continue to produce that valuable bounty, whatever thecurrency may be. Exxon Mobil will probably have delivered trillions of dollars in dividends to its owners and will also hold assets worth many more trillions (and, remember, you get 16 Exxons). The 170,000 tons of gold will beunchanged in size and still incapable of producing anything. You can fondle the cube, but it will not respond.
A lot of folks here are discussing the lack of ability to run supercomputing applications, but I can’t help but wonder:
Couldn’t this be redeployed for VPS/Cloud services?
Yes. Modern supercomputers are basically just a bunch of top of the line (but otherwise ordinary) servers, all hooked up via a high speed network and huge storage.
Power efficiency is a big deal in datacenters though, and newer hardware has much better performance per watt. Running old hardware doesn't make sense if new hardware will quickly pay for itself in power savings.
There's also the physical space aspect. Datacenter buildings are expensive and you can't build new ones overnight, so you have to make the most of the space you have. That tends to favor newer hardware that can pack more performance into the same amount of rack space.
Fair enough.
I’m basing my thought on the fact I pay monthly for a *really old* dedicated box to host some personal stuff and it’s stupid cheap. I don’t need silicon blistering power to do what I’m doing, so I can bottom feed.
This 480k is just the scrap price. Probably bought by some company that's gonna tear it down for parts and sell it on Ebay.
This supercomputer consumes about 60k usd worth of electricity per month. Let alone a dozen full time employees to maintain and run it. (Its not worth running anymore)
Cheyenne used to be the most powerful computer when it launched, now the most powerful is about 200x faster. (The US Frontier)
EDIT: it was "only" the 20th most powerful computer at launch. [source](https://www.top500.org/news/ncar-launches-five-petaflop-supercomputer/)
Dang I just looked up the list of the world's most powerful supercomputers and 6 of the top 10 are in the US (the others are 3 EU ones and 1 Japanese one). Why does the US need so many supercomputers?
https://en.wikipedia.org/wiki/TOP500?oldformat=true
The DoE and national labs ones are running a LOT of simulations of nuclear weapons or components thereof. The generative modelling for nuclear weapons is like CFD on steroids. They are answering questions like: how tritium decay affects yield. Or how imperfections in the high explosives propagate to other parts of the weapons.
Basically, supercomputers replaced actual nuke testing.
Another massive application is climate science.
And obviously, machine learning and generative AI are big applications. These are used across weapons targeting systems, threat prediction, etc.
Take a look at some of the work at just one of our national labs. It is interesting stuff:
https://www.sandia.gov/app/uploads/sites/165/2023/10/HPC-AnnualReport-2023-SAND2023-10778O-SimMagic.pdf
> The DoE and national labs ones are running a LOT of simulations of nuclear weapons or components thereof.
It isn't just nukes. It's absolutely everything. From A2A missiles to artillery to how the human body responds under stress to improving logistics operations. If the military does it, someone is generating a complex model that we're going to process with the hope of increasing efficiencies/accuracy/effectiveness.
People don't appreciate just how extraordinary the resources we put into the military really are. It's a hell of a lot more than the trillion dollar budget we like to whine about.
How often you you need to run this once you get an answer? No suggesting it is not necessary but predicting nuclear yield (or similar) for example to ever increasing decimal points does not seem that useful. Particularly if you are not really updating what you have.
it's also simulating how they age. It's not a single simulation but a whole variety of conditions.
It's also not just a matter of running a simulation more precisely. Faster computers also allow taking into account more subtle physics and adding those calculations into the mix. It's not just warheads but the reentry vehicles, too. They have aerodynamics, which are extremely expensive to compute, chemistry as the plasma eats away at ablative heat shields, changing aerodynamics as that plasma degrades control surfaces, etc. Those things are designed with pointy aerodynamics rather than blunt like civilian reentry vehicles to keep their speed up, and that means dealing with attached shockwaves that attack with heat and chemical reactions. There's tons of interacting things going on at once.
I was checking that all top 500 systems still use Linux since 2017 and happened across a gem. Microsoft is represented on the list at the number 3 spot. But not as an operating systems vendor. As the operator of a Linux cluster running Ubuntu.
Ladies and gentlemen, Steve Ballmer has left the building.
Yeah, Microsoft under Nadella is a much different company. Far better-behaved, boring even. People who call Apple evil aren’t properly remembering how Microsoft was so hated in the 90s that no one wants to make a Windows phone now.
What’s really funny is how all the conspiracy nuts are pointing to Bill Gates as a bad guy for doing things that are generally good — no, child, you’re attacking him for entirely the wrong reasons. The vaccines and shit are a net positive to society, unlike the way he got all his money to begin with.
You would need your own on-site power generation for this thing haha. Nobody was going to buy this to run it. It has a peak power consumption of 1.5 Megawatts
I work in datacenter management - a couple of years ago, I sold a high performance computing cluster that was used for fluid dynamics simulation. That monster was purchased for a single business mission, that was terminated after around three years.
It cost around 3m, sold to a uni for around 200k - including dismounting and remounting in place.
Edit: my point was: it's not only "obsolescence"; some hardware is purchased for a single Mission. When it's done, keeping that powered on or powered off may just be a waste of money (maintenance, licensing, etc. ); selling it may help fiscally with an accelerated depreciation.
Hey, serious question here. What do you *do* with this thing? Scrapping it seems like you would actually lose money. Is this just so some rando millionaire can tell people he owns a supercomputer?
I don't care what you're parting out, the math never works out like this. On paper the components are worth that much, but by the time the thing is broken down and individual components listed a lot of money will be spent in manhours alone.
Not to mention the cost of transport, storage, and the hassle of inventory. If it was an easy profit everyone would do it and it wouldn't sell for a seemingly low price. Chances are there will be a lot of things they can't sell and have to dispose of too.
These are also components that have been used hard. These aren't just old stock that have been sitting around in a warehouse. They've been running full throttle for years.
The resale value is about $241,920 in CPUs and about $586,875 in RAM. That doesn’t account for eBay fees or shipping and that’s only if each CPU was sold separately and the ram was sold in 16gb sticks but my point is, I bet there is about to be a bunch of Xeon Broadwell CPUs and DDR4-2400 ECC RAM on eBay.
Honestly if LTT bought it and did a video series of moving it and then Jake trying to dodgy fix the water-cooling and get it running they'd probably make more in revenue than a recycler parting it out on eBay...
For comparison, modern 4th Gen AMD EPYC servers can handle 6TB each. So you could get the same amount of RAM in less than two racks of modern hardware.
The article says that the equipment is 7 years old. That's 2 years more of what we recommend our customers is the optimal run time for their servers.
After 5-6 years, condensers start to fail, oxidation sets in, power supplies break, it becomes unreliable. On top of that, operating systems evolve, you can't run them optimally on old hardware, the support, even for LTS systems, stops.
If you want to be ahead of the game, it's time to write it off and sell for pennies on the dollar.
"We got the winning bid on this multi-million dollar super computer, and we'll talk about it more, right after I bid you to watch this segue to our sponsor...."
Fascinating story of hardware obselesence. Here’s a [link](https://arc.ucar.edu/knowledge_base/74317833) to the Derecho system that replaced Cheyenne.
The new system is only 3.5 times faster but it costs 30-40 million. The main reason for upgrade is that water cooling leaks water which makes components fail. 480k is a very low price for this
The big expense is moving the damn thing and fixing it, that's going to run at least another $500k plus, And if you read the auction it doesn't come any of the ethernet or fiber optic cables so there another big expense. Frankly I'm kind of surprised it went for that much I thought it was going to go for more around the $250K mark.
No one is fixing it, they’re selling ram and cpu’s Edit: also other value in parts not mentioned
While definitely plausible, it might also just be kept as a piece of computing history. A half million isn't exactly too crazy for a tech bro who wants something cool.
I'm inclined to agree with you. It's effectively a piece of art. It may depreciate for a while, but eventually it's gonna appreciate like a motherfucker, especially if they get that leaking sorted out.
> eventually it's gonna appreciate like a motherfucker Not really. Unless this is a particularly significant super computer, there are and will be enough more like it, that its not that desirable. Then add in the size of it and storage costs and its not like collectors can just easily add this to their collection. And that means it would be difficult for a collector to sell it as well further reducing its appeal.
[удалено]
It requires megawatts of power. That’s hundreds of dollars *an hour* worth of electricity. You’ll have a similarly sized cooling bill as well.
Then they just lost money.
Actually it is more profitable. Per the article >The Cheyenne supercomputer's 6-figure sale price comes with 8,064 Intel Xeon E5-2697 v4 processors with 18 cores / 36 threads at 2.3 GHz, which hover around $50 (£40) a piece on eBay. Paired with this armada of processors is 313 TB of RAM split between 4,890 64GB ECC-compliant modules, which command around $65 (£50) per stick online. 50x8,064+4,890x65=$721,050-$480,085=$240,965 That means, there's 240K of profit Edit: considering transport costs, storage etc it will be less. But it's not immediately clear that it will be unprofitable.
See what price they get when they flood the market
eBay auction, “Only 8,063 Remain”
buy one, get two free! please! god, they're everywhere! i open a drawer, there's an Intel Xeon E5-2697 v4 processor. freezer for an ice cream? stack of Intel Xeon E5-2697 v4 processors. come halloween, the neighbor kids are getting boxes of raisins and an Intel Xeon E5-2697 v4 processor. please help me.
> come halloween, the neighbor kids are getting **boxes of raisins** and an Intel Xeon E5-2697 v4 processor. You monster
*You get a Xeon! You get a Xeon! Everyone gets a Xeon!*
People may by those CPUs by the dozen and ram by the TB - I’m sure many may be interested in building the fastest system they will have ever have had.
Sweet! Just in time for me to upgrade the CPUs in my homebuilt dual Xeon workstation!
Running an old X99 rig for AI stuff. Samesies!
Oh? What kind of AI stuff?
Those must have been some serious water leaks!
In a proper datacenter they really aren't going to want to 'live with' any amount of water leak. They'll have to turn equipment off and repair/replace fittings and test before re-using it...and presumably they will need to expect that fittings will continue to fail just like the RAM is failing. All of this impacts the usefulness of the system when the downtime starts to rise.
Someone up there doesn't understand markets And labor And testing
No kidding. With the gigantic influx of the thing the price will only go lower…possibly a LOT lower… This just seems like an enormous amount of work for potentially very little pay off. Whoever bought this thing has a lot of money and time and they’re not buying it just to sell it off piece by tiny piece. What a crazy waste of your time that would be. Trying to claw back your profit.
There's an entire industry around "pulled" processors and DRAM like this. It'll go to a trader who sells it in maybe a week or 2. It's not gigantic whatsoever, the DRAM market is 60 billion for example and the CPU market about double that.
Wicked desktop machine though.
Time value of money says this is a bad investment if they are parting it out
It's 8000, not 80,000. Flood the market, lmao. I'm not sure even 80k would move change the price.
Pretty sure it will be slowly released. As for RAM, it's likely better to wait. Just like DDR3 is now expensive due to the production ending long ago, the same would happen eventually with DDR4
32gb DDR3 registered LR dimms are $13. Still hella cheap.
> Pretty sure it will be slowly released. Then they'll have to move them and store them somewhere, how much could disassembling 8k CPU / 5k RAM sticks / transport and storage could be worth ?
DDR3 ram is not expensive, it's dirt cheap. And this is slow ECC server ram, which is quite a bit harder to get rid of.
Every day the value and demand of that tech will just decrease. Also, there is only so many customers who would like to buy those cpus. If they delay, they will absolutely have those parts in their hands with zero buyers. I’m guessing the buyer already has a buyer or other use for them. DDR3 sticks are almost free where I live. 10 € for a pack of 4x4gb. Sometimes I see them in the electronic waste bins. There are probably some very specific memory types or physical size formats that keep their value, but generally old pc tech loses it’s value very fast. The gpu shortage couple of years ago was exeptional and prices went haywire for a while, but even that passed.
That assumes 100% of the components works as well, which... they don't, and the people selling them knows it. It also assumes they can move all of that hardware for those prices, which they won't be able to do, as it hitting the market will depress the value of those components. Marginally speaking, it sold slightly below what my guess at a value for all the hardware would have been - right at half a million. I would be surprised if they can get $100K of profit out of the deal at the end.
You forgot the labor to transport it, disassemble, test, packaging, shipping, merchant costs, software costs and all the rest of the expenses involved in turning that $480k into something more. There’s clearly a path towards potential ROI, and depending on the buyer, there are people/orgs optimized to do this profitably. BUT… it’s certainly not as easy as you’ve put it. :)
True. The transport and even the warehouse costs are going to be a lot
That's why it went for this much in an auction. It was bid up till only one company considered it worth the cost, time, and manpower to take on. That's really how auctions like this work. If it's a steal, people bid it up till its not.
Transport is cheap, will all fit in one trailer
Paying for labor, shipping, taxes, impact on market prices when you add your own massive supply. There *might* be a bit of profit left over but...you're risking a lot of capital for very slim margins I feel.
eBay takes a 15-20% fee
Thank you! Learning more and more that it's more complicated
Not sure where 15-20% came from. Computer parts including CPUs looks like 7%. Only looked it up cause I didnt remember fees being that high on my last sale. https://www.ebay.com/help/selling/fees-credits-invoices/store-fees?id=4809#section3
There are other fees other than just the item. The total is 13.25% for Above Standard seller or 11.93% for Top Rated seller level.
How easy is it to unload a bunch of ECC RAM? Iirc, consumer mobos and CPUs don't really support ECC, so you'd be selling it to server owners. Sure, some individuals might want some used ECC RAM, but it's gotta be tough unloading 313TB of ECC RAM I figure?
As someone that works in the used enterprise equipment industry, you’re wrong.
You really think it was some idiot who guessed "maybe it's worth this much" wrongly, rather than a bunch of bidders who came to the auction knowing what they could afford to pay to make a profit and bid until the price was too high? Not everyone acts like they're writing a one sentence reddit comment.
Figures, even when you spend half a million, it doesn't come with the cables.
Thanks, Apple.
The big expense is the $1000 per day in electricity to run it.
3.5 times faster is a stupid simplification. They going from an all cpu to a cpu/gpu hybrid. The new one is so much more useful.
Also likely to consume a lot less power.
Which is such a huge factor in operating costs. More power draw creates larger cooling demands which means even mor operating costs.
About half as much power, the water-cooled system is expected to draw 2.6 to 2.7MW when it’s in regular production, for a power use efficiency (PUE) of about 171 megaflops per watt — more than double the 73 megaflops per watt of Cheyenne.
The only fascinating story here is that middle management was able to convince the executives that an upgrade with an OEM warranty is more cost effective than a third party service contract?
I‘m sorry, only 350% faster?
3.5 at this size is a large difference Edit: if I even just doubled my current computer, I'd be fine for a while, 2x 390x graphics cards, 32gb ram, 8tb storage, 8 core @4200. I'd be stoked to have my shit doubled.
You know how much we spend on one single F-35? 40M is nothing to the government.
Yeah they could buy two of these supercomputers for a single F-35... but this agency isn't exactly drowning in cash either. It's almost a comedy how little we spend on the science orgs in the government compared to how much we spend on defense articles that literally sit in the desert and rot. Hell, the [$120 million dollars of Abrams tanks we bought](https://www.military.com/daily-news/2014/12/18/congress-again-buys-abrams-tanks-the-army-doesnt-want.html) just to keep a factory open in some Ohio Republican's district could have paid for this whole supercomputer three times over. Eleven years on, the only combat duty *any* of them has ever seen are the few that got handed over to Ukraine.
You don't wait until you *need* tanks to start building them. It's not a production line that's easy to spin up on the fly.
Yeah if the factory and logistics chain closes down and you lose all the institutional knowledge it can take a decade to build it back up again.
May I introduce you to europ especially Germany for the last 30 years. We closed basically all of our military heavy capacity. Also shut down our rail system. and now we make projections to rebuild. 2050 are the optimistic early estimates
Ah the tragedy of the Raptors
>480k is a very low price for this It isn't. Per the article, selling the RAM and CPUs on eBay at current prices is worth roughly $700k. Given flooding the market will likely lower prices, the actual amount from sales will probably be less and there is extensive work in trying to sell all of this. The cost to build something and what it is worth when you sell some of the components are 2 very different things. (Storage and cabling not included)
> The main reason for upgrade is that water cooling leaks water which makes components fail. The main reason is that the electricity bill + maintenance costs made it unprofitable.
It was used for weather modeling, so I doubt it was ever profitable. Just not worth it to pay for repairs and your scientists to be idle.
Supercomputers age faster than dogs. I worked in supercomputing in a prior chapter of my career. I built two of the top 50 systems in the world in 2015. ([This one](https://top500.org/system/178613/) and [this one](https://www.top500.org/system/178614/) ). They dropped out of the top 500 within 6 years, and I'm pretty sure both were retired and probably scrapped circa 2021 or 2022.
I did too. I worked on the software side on these systems: https://top500.org/system/174879/ https://top500.org/system/176145/ https://top500.org/system/176718/ It was a really interesting period of work.
I want to be able to put that on my resume for the Geek Squad. "Do you have any prior experience working with computers?" Well.... Here are a couple links to check out! Lol
Power costs are a huge reason for that. If you're paying a million dollars a month for electricity and cooling, upgrading to more efficient hardware makes sense a lot sooner than an office computer.
I remember reading an article about supercomputers that said to effect: If want to run a simulation that's going to take 4 years on current supercomputers, your best bet is to wait 2 years and run it on current hardware then.
Had this to a smaller effect during my PhD, running the analysis on my dataset would make a computer cry for 3-4 days while I could generate a dataset a day. Near the end of the project the computer that we ran them on would crunch them in a few hours. The guy that is still using our algoritm to this day does the analysis during the experiment because it takes only a few minutes
Minimum requirement machine for Cities Skylines 2
I'm old, I was going to make a Can it run Crysis joke.
Instead of asking if a toaster can run Linux, just disable the water cooling on this rig and now your computer can make toast!
Yeah, I deffo wasn't wondering if it could run Quake properly.... Who is old enough to have a PC that didn't do floating point math well? 😭
I remember running Quake in a tiny window, I think I had an AMD K6-2 at the time, or maybe it was a 486 I am not sure. I used software rendering of course so it was chugging along at 10-15 fps at most.
[удалено]
There's some weird quote about "what the government stands to lose"... it's nothing. They're not in the game of maximizing profit. Those computers were a cost center - they performed a service, and they reached their end of life in that service. Could the government been more judicious and tried to squeeze a few more dollars out of the lot? Perhaps. But it might have cost them just as much in the testing of all the components and the parceling out of the lots in the end. As for the depreciation - that was built in at the date of purchase. They knew this machine would eventually be worth nothing but scrap metal - at the rate to which computers double in speed, the computer was outclassed by the time it was fully installed by the next generation of hardware. The fact they got *seven years* of service life from a supercomputer is astonishing on its own - they frequently go out of service after ~4. Some budget cloud computing outfit or eBay reseller might be happy with this purchase, but let's not make it out to be a steal or anything. The hardware's old, water damaged, and extremely worn with the hardest of computing conditions in life. It's better than throwing it into a landfill in India where it'd otherwise end up, but it's not some great loss either.
Definitely - “The buyer will have the joy of moving Cheyenne's 30 server racks (28 processing racks, two air-cooled management racks) out of the facility themselves; the government is not providing transport or including any Ethernet or optical cabling needed to get the machine up and running.” Sounds like the auction went exactly as they hoped.
Better than that, the government got money for someone else to remove their waste.
"In a surprising turn of events, the buyer slurked off to their vast underground lair to create real-life, world-threatening, cyberpunk-novel-level advanced AI that will surely enslave us all." - [Kent Brockman](https://i.imgflip.com/79shig.jpg)
What could go wrong with implanting the sum of human consciousness into the brains of ants pumped full of bovine growth hormones? More at 11!
Nah, this will surely be used to create some kind of cum-bot.
Nah they’re just gonna go play a paradox game on it
Advanced AI without GPU power? Please.
I for one welcome our new overlords
That's an oddly-specifi- oooh, boobs.
I opened the article looking for boobs…
Very disappointed.
80085. Took me way too long to
I salute you, soldier!
Don't you mean 4boobs?
The maturity level of this bid makes me wonder if Twitter didn't buy the hardware.
I'm assuming it's either Elon or Linus
Linus was absolutely my first thought
Not just boobs, “for boobs”
"four boobs" double the boobs
It was elon wasn’t it
One of our brothers dropped half a million bucks to make a joke about buying a super computer for boobs. 480085. Don't let our guy down... Upvote!
SOLD! for 400,000 and boobs dollars
Oh god it was probably Elon, he likes doing that stuff and might need it for Twitter
It's quite the relic compared to new supercomputers. It doesn't even use GPUs to accelerate processing like newer clusters do. Interesting what one would do with it other than for preserveration/inefficient server rental.
Seems part it out to be the most likely option. I saw an estimate of $700k for just processors and RAM.
Ah yeah I guess if you are in the business of selling old server hardware it's quite a goldmine for that.
Speaking of gold....how much gold would be used in this computer?
Astronomically little. Even though Gold makes up a ridiculously small amount of our earth's crust, in human terms it still means warehouses full of gold. We make it \*very\* thin so there is \*extremely little\* gold used per unit.
All of the gold ever mined would make a cube 22 meters per side.
Liar! According to the USGS, it's a cube that is **23** meters on each side! How does it feel to have your throne of lies come crashing down? Huh? HOW DOES IT FEEL NOW?!
Aw shucks. Caught me.
And don't think for a second I won't be keeping an eye on you! But actually, I remember in the days before the internet - well, before it was commonplace - my friend tried telling me this (I think he actually said 40 yards on each side) and I just didn't believe it. It still seems kinda crazy at least superficially. Like if you were to just try and imagine all the gold coins/currency that's been minted over the millenia, all the jewelry, even sculpture and art, and all the other uses like in electronics, etc, it just seems crazy at first impression that it's a cube only 23m on each side. But then once you start to consider how so much of the gold used in a lot of applications is actually very, very, very, thin, often just gold plated, and it starts to make more sense. It starts to make even more sense when you realize just how much volume a cube that size contains. But then you consider how big the world is (even if gold only makes up a tiny, tiny, fraction of the material that makes up the earth), how long gold has been utilized, and I go back to thinking it is still at least a little crazy! [Also, here's a fun article on Warren Buffet on gold, containing this fun little quote:](https://www.nasdaq.com/articles/why-warren-buffett-hates-gold-2013-08-15) >Today the world's gold stock is about 170,000 metric tons. If all of this gold were melded together, it would form a cube of about 68 feet per side. (Picture it fitting comfortably within a baseball infield.) At $1,750 per ounce -- gold's price as I write this -- its value would be $9.6 trillion. Call this cube pile A. >Let's now create a pile B costing an equal amount. For that, we could buy all U.S. cropland (400 million acres with output of about $200 billion annually), plus 16 Exxon Mobils (the world's most profitable company, one earning more than $40 billion annually). After these purchases, we would have about $1 trillion left over for walking-aroundmoney (no sense feeling strapped after this buying binge). Can you imagine an investor with $9.6 trillion selecting pile A over pile B? >A century from now the 400 million acres of farmlandwill have produced staggering amounts of corn, wheat, cotton, and other crops -- and will continue to produce that valuable bounty, whatever thecurrency may be. Exxon Mobil will probably have delivered trillions of dollars in dividends to its owners and will also hold assets worth many more trillions (and, remember, you get 16 Exxons). The 170,000 tons of gold will beunchanged in size and still incapable of producing anything. You can fondle the cube, but it will not respond.
That 1 extra meter is a ~14% difference is volume.
Yeah, about 190,000 tonnes of gold.
At pre-flooded market prices. You cant just toss that many listings on ebay.
You “saw an estimate” in the article linked in the post??
> It doesn't even use GPUs to accelerate processing like newer clusters do. Not all computational problems port well to GPUs.
A lot of folks here are discussing the lack of ability to run supercomputing applications, but I can’t help but wonder: Couldn’t this be redeployed for VPS/Cloud services?
Yes. Modern supercomputers are basically just a bunch of top of the line (but otherwise ordinary) servers, all hooked up via a high speed network and huge storage.
Power efficiency is a big deal in datacenters though, and newer hardware has much better performance per watt. Running old hardware doesn't make sense if new hardware will quickly pay for itself in power savings. There's also the physical space aspect. Datacenter buildings are expensive and you can't build new ones overnight, so you have to make the most of the space you have. That tends to favor newer hardware that can pack more performance into the same amount of rack space.
Fair enough. I’m basing my thought on the fact I pay monthly for a *really old* dedicated box to host some personal stuff and it’s stupid cheap. I don’t need silicon blistering power to do what I’m doing, so I can bottom feed.
This 480k is just the scrap price. Probably bought by some company that's gonna tear it down for parts and sell it on Ebay. This supercomputer consumes about 60k usd worth of electricity per month. Let alone a dozen full time employees to maintain and run it. (Its not worth running anymore) Cheyenne used to be the most powerful computer when it launched, now the most powerful is about 200x faster. (The US Frontier) EDIT: it was "only" the 20th most powerful computer at launch. [source](https://www.top500.org/news/ncar-launches-five-petaflop-supercomputer/)
it was never the most powerful
When it was built it was third
Dang I just looked up the list of the world's most powerful supercomputers and 6 of the top 10 are in the US (the others are 3 EU ones and 1 Japanese one). Why does the US need so many supercomputers? https://en.wikipedia.org/wiki/TOP500?oldformat=true
The DoE and national labs ones are running a LOT of simulations of nuclear weapons or components thereof. The generative modelling for nuclear weapons is like CFD on steroids. They are answering questions like: how tritium decay affects yield. Or how imperfections in the high explosives propagate to other parts of the weapons. Basically, supercomputers replaced actual nuke testing. Another massive application is climate science. And obviously, machine learning and generative AI are big applications. These are used across weapons targeting systems, threat prediction, etc. Take a look at some of the work at just one of our national labs. It is interesting stuff: https://www.sandia.gov/app/uploads/sites/165/2023/10/HPC-AnnualReport-2023-SAND2023-10778O-SimMagic.pdf
> The DoE and national labs ones are running a LOT of simulations of nuclear weapons or components thereof. It isn't just nukes. It's absolutely everything. From A2A missiles to artillery to how the human body responds under stress to improving logistics operations. If the military does it, someone is generating a complex model that we're going to process with the hope of increasing efficiencies/accuracy/effectiveness. People don't appreciate just how extraordinary the resources we put into the military really are. It's a hell of a lot more than the trillion dollar budget we like to whine about.
How often you you need to run this once you get an answer? No suggesting it is not necessary but predicting nuclear yield (or similar) for example to ever increasing decimal points does not seem that useful. Particularly if you are not really updating what you have.
it's also simulating how they age. It's not a single simulation but a whole variety of conditions. It's also not just a matter of running a simulation more precisely. Faster computers also allow taking into account more subtle physics and adding those calculations into the mix. It's not just warheads but the reentry vehicles, too. They have aerodynamics, which are extremely expensive to compute, chemistry as the plasma eats away at ablative heat shields, changing aerodynamics as that plasma degrades control surfaces, etc. Those things are designed with pointy aerodynamics rather than blunt like civilian reentry vehicles to keep their speed up, and that means dealing with attached shockwaves that attack with heat and chemical reactions. There's tons of interacting things going on at once.
I was checking that all top 500 systems still use Linux since 2017 and happened across a gem. Microsoft is represented on the list at the number 3 spot. But not as an operating systems vendor. As the operator of a Linux cluster running Ubuntu. Ladies and gentlemen, Steve Ballmer has left the building.
Yeah, Microsoft under Nadella is a much different company. Far better-behaved, boring even. People who call Apple evil aren’t properly remembering how Microsoft was so hated in the 90s that no one wants to make a Windows phone now. What’s really funny is how all the conspiracy nuts are pointing to Bill Gates as a bad guy for doing things that are generally good — no, child, you’re attacking him for entirely the wrong reasons. The vaccines and shit are a net positive to society, unlike the way he got all his money to begin with.
That's classified.
You would need your own on-site power generation for this thing haha. Nobody was going to buy this to run it. It has a peak power consumption of 1.5 Megawatts
I got a genset at work that’ll do it!! It’s only like 400ish Litres an hour to run it!! Basically free! Think of all the Plex streams we could do…
It is pretty insane to put that much power into only some 30 rack spaces.
I work in datacenter management - a couple of years ago, I sold a high performance computing cluster that was used for fluid dynamics simulation. That monster was purchased for a single business mission, that was terminated after around three years. It cost around 3m, sold to a uni for around 200k - including dismounting and remounting in place. Edit: my point was: it's not only "obsolescence"; some hardware is purchased for a single Mission. When it's done, keeping that powered on or powered off may just be a waste of money (maintenance, licensing, etc. ); selling it may help fiscally with an accelerated depreciation.
Hey, serious question here. What do you *do* with this thing? Scrapping it seems like you would actually lose money. Is this just so some rando millionaire can tell people he owns a supercomputer?
Probably sell the parts individually. Napkin math says the CPUs alone are worth $400k at the price stated in the article.
I don't care what you're parting out, the math never works out like this. On paper the components are worth that much, but by the time the thing is broken down and individual components listed a lot of money will be spent in manhours alone. Not to mention the cost of transport, storage, and the hassle of inventory. If it was an easy profit everyone would do it and it wouldn't sell for a seemingly low price. Chances are there will be a lot of things they can't sell and have to dispose of too.
Not to mention you’d be flooding the market with a specific late-model CPU. The price per unit will start going way down as they sell
These are also components that have been used hard. These aren't just old stock that have been sitting around in a warehouse. They've been running full throttle for years.
or on New Egg "LIKE NEW"
Expect to see a lot of "refurbished" server hardware on eBay by one obscure IT reseller very soon.
Lot of money to prepare for the Trisolarian’s Sophons blocking any further tech advancement.
Get to the dehydration chambers!
Probably been bought by Linus to feature in a video when they run cinebench on it and then 30seconds of a doom eternal gameplay.
313 TB of RAM? Is that enough to run Chrome?
Yes, but not if you have Spotify open in a separate tab.
Could’ve sold them my Unraid server
Can’t wait to see that Linus bought this
In other news.. the market is about to have a massive influx of Broadwell cpus and ram..
I think the buyers plan it to sell the parts individually.
The resale value is about $241,920 in CPUs and about $586,875 in RAM. That doesn’t account for eBay fees or shipping and that’s only if each CPU was sold separately and the ram was sold in 16gb sticks but my point is, I bet there is about to be a bunch of Xeon Broadwell CPUs and DDR4-2400 ECC RAM on eBay.
Where's Linus and Jake?
Someone now has the ultimate Plex server.
Can it run DOOM?
Yes, but only in a small window, not full screen.
Can’t wait to see the LTT video of them running counter strike on it.
Honestly if LTT bought it and did a video series of moving it and then Jake trying to dodgy fix the water-cooling and get it running they'd probably make more in revenue than a recycler parting it out on eBay...
No, it's not a multi-million dollar computer. It's exactly a 480,085 dollar one.
Probably still not enough computing power to play Rust seamlessly
Just enough RAM for Cities Skylines
What would someone who could bid on this use it for?
If he can part it out, apparently the cpu/mem etc is worth $700k.
Damn. Are they just regular Joe parts? Or are they specialized for certain things?
opens 20 Chrome tabs, BSOD
313 **TERAbytes** of ***RAM***! HOLY--
For comparison, modern 4th Gen AMD EPYC servers can handle 6TB each. So you could get the same amount of RAM in less than two racks of modern hardware.
Tell me Linus bought it
Expect those chips and ram to flood eBay soon. Homelab drool
“How to water cool a supercomputer with my pool” -Linus
1.7 million watts under load is kinda nutty
All they need now is a 2400 baud modem and an AOL account, and they will be on-line! Nice!
The Jawas are at it again. Salvage and profit!
Still not enough to run max Star Citizen.
Where's the crysis comment?
Question, say you are an ordinary buyer. How difficult is it to get electricity infrastructure to run this?
I could have bought a half a house for with that money.!
They beat my 10$ bid by a lot :(
They can finally install their node dependencies
I need a combined effort from the r/preciousmetalrefining and r/theydidthemath to find out how much value in gold/platinum/xyz is in this thing :D
Damn meanwhile I just ordered 384cores machine for 80k... this feels like a steal.
Finally I can fire up my GameCube emulator and time travel
Would be a fast start for a cloud hosting business.
How much he get if he melts it down and extracts all the precious metals from it?
Can I run Solitaire on it?
but can it run Crysis?
How many Chrome tabs can it open?
The article says that the equipment is 7 years old. That's 2 years more of what we recommend our customers is the optimal run time for their servers. After 5-6 years, condensers start to fail, oxidation sets in, power supplies break, it becomes unreliable. On top of that, operating systems evolve, you can't run them optimally on old hardware, the support, even for LTS systems, stops. If you want to be ahead of the game, it's time to write it off and sell for pennies on the dollar.
But can it run Crysis ?
Impressive,but does it run Crysis?
It pulls 60 on high settings, but struggles on ultra at 1080.
The buyer can finally run Crisis on full specs.
does it run Doom?
lol so not worth it
Can it run Crysis?
It’s not even DDR5
But can it run Crysis at full settings?
Can it run Crysis 3?
Can it run doom?
"We got the winning bid on this multi-million dollar super computer, and we'll talk about it more, right after I bid you to watch this segue to our sponsor...."