interconnects

The interconnects and the overall use of a supercomputer is drastically different than what you have on EC2.

If the data is mission critical to your company, you consume the feed via private leased lines or interconnects.

This is why using plasmonics in ICs is an active field of research; the hope is to be able to use optics for long distance interconnects inside the processor.

Unlikely; the limiting factor is already distance - poor scaling from interconnects (wires) already means that we can't have all that much global state.

Of course, if they start selling 1 Gigabit Service, that's a lot of switch ports/chassis/routers/interconnects/etc/etc...

Exposed interconnects are 10 gige, so not the best, but like you said, scale requires designing for minimizing communication to beginwith.

Unfortunately, you end up having to make the interconnects longer to accommodate the additional redundant components, which ends up slowing everything down.

Electromigration will move atoms on the interconnects between transistors and eventually cause an open.

When you read about HPC you'll hear a lot about high-speed interconnects, and using DMA-based communication instead of TCP for higher performance.

I think it's economics, GPU's are sold by the million, super computers interconnects are sold by the thousands.

In Denmark this is mostly managed by the interconnects with neighboring hydro-heavy grids.

It's still widely used in telco networks, transporting control messages to/from your mobile device within the core network, and in interconnects between carriers.

I don't think "really fancy interconnects" makes a cluster not commodity, since the original post in this thread is about supercomputer processors.

To say that the 3D engine is "one of the most beautiful components" of a game, is similar to saying that the optical fiber interconnects are "one of the most beautiful components" of a server rack.