I bet manpower costs are significant as well. How many people are needed to run this thing? You probably need engineers with an esoteric set of skills to put it back together and manage it which would not be cheap.
Edit: I looked it up, it is running SUSE Enterprise Linux, so maybe management isn’t as specialized as I expected.
Kind of, you would use a deployment node to manage the individual blades, they are running really specialized software that is basically useless without the management nodes. It wouldn’t be difficult to spin it up (Terascale would have it ready to batch out jobs within a few hours) but you are going to need to engineer your building around it to even get that far. Your foundation needs to support multiple tons of weight, be perfectly level, be able to deliver megawatts of power, remove megawatts of heat (it is water cooled, so you need to have infrastructure and cooling towers to handle that), and you need to be able to get it into the building to begin with. I have worked on this system a few times, just moving it would literally cost upwards of 7 figures. The computer is pretty easy to use, it’s all of the supporting infrastructure that will need a literal team of engineers. I could (and have, kind of) spin the machine up to start crunching data within a day on my own. Fuck moving it, and double fuck re-cabling it. Literal miles of fiber in those racks.
You do literally pop in an image that is pre-configured in and it deploys to everything at once. That’s probably the easiest part of the whole setup.
I bet manpower costs are significant as well. How many people are needed to run this thing? You probably need engineers with an esoteric set of skills to put it back together and manage it which would not be cheap.
Edit: I looked it up, it is running SUSE Enterprise Linux, so maybe management isn’t as specialized as I expected.
It may be running SLED, but just imagine all the specialized, tweaked af code running on top. They didn’t just pop in a LiveCD and click “Install”.
No, they probably had to pop the live CD into each node individually and click “instal”. Then run a script on each one to join it to the cluster.
Kind of, you would use a deployment node to manage the individual blades, they are running really specialized software that is basically useless without the management nodes. It wouldn’t be difficult to spin it up (Terascale would have it ready to batch out jobs within a few hours) but you are going to need to engineer your building around it to even get that far. Your foundation needs to support multiple tons of weight, be perfectly level, be able to deliver megawatts of power, remove megawatts of heat (it is water cooled, so you need to have infrastructure and cooling towers to handle that), and you need to be able to get it into the building to begin with. I have worked on this system a few times, just moving it would literally cost upwards of 7 figures. The computer is pretty easy to use, it’s all of the supporting infrastructure that will need a literal team of engineers. I could (and have, kind of) spin the machine up to start crunching data within a day on my own. Fuck moving it, and double fuck re-cabling it. Literal miles of fiber in those racks.
You do literally pop in an image that is pre-configured in and it deploys to everything at once. That’s probably the easiest part of the whole setup.
I tried hard to oversimplify. Thanks for spoiling it.
Of course. I was obviously referring to what it takes to operate it after that. Not to mention how complicated setting that whole mess up is.
Obviously not. In 2017, they would have used a live USB thumbdrive instead of a CD.
Yup, most of these are just a lot of relatively normal hardware put together into one system.