Thursday Threads: Cloud Computing and Data Centers -- Amazon, Facebook, and Google
This week's DLTJ Thursday Threads is about data centers -- those dark rooms with all of the blinking lights of computers doing our bidding. Data centers hit the mainstream news this week with the outage at one of Amazon's cloud computing clusters. And since computers and their associated peripherals consume a lot of energy, researchers are proposing to run data centers on renewable energy. And finally Facebook and Google release separate videos that give glimpses into how large data centers are run.
Feel free to send this newsletter to others you think might be interested in the topics. If you are not already subscribed to DLTJ's Thursday Threads, visit the sign-up page. If you would like a more raw and immediate version of these types of stories, follow me on Mastodon where I post the bookmarks I save. Comments and tips, as always, are welcome.
Amazon EC2 Outage Hobbles Websites
Failures of Amazon's Elastic Compute Cloud service -- think of it as renting virtual computer servers somewhere out there on the internet -- last week caused major internet sites to shut down. As of this writing, the root cause analysis hasn't been published, but signs are pointing to a cascade of events starting with a minor failure that snowballed into system overload as the rented servers tried to restart themselves in other areas of Amazon's cloud capacity. The questions being raised though are leading to a darkening of the puffy white cloud computing promise. Ultimately, though, use of computing in the cloud seems to be a trade-off where you can save money by not owning your own computing infrastructure with the downside that you don't have as much control when something goes wrong.
Far-flung Data Centers Could Use Otherwise Unharvestable Renewable Energy For Computation
The second thread comes by way of MIT Technology Review and points to a paper by Sherif Akoush, Ripduman Sohan, Andrew Rice, Andrew W. Moore and Andy Hopper -- all of Cambridge University called Free Lunch: Exploiting Renewable Energy For Computing, to be presented at the USENIX-sponsored the 13th Workshop on Hot Topics in Operating Systems next month. The "Free Lunch" part comes from using renewable energy sources at these various locations to power data centers where compute jobs are shuffled around the locations depending on the available energy -- and consequently computing capacity -- at each center. A neat idea, and one that is probably valuable for compute-intensive jobs like video conversion and data mining.
What Goes Into Running Large Data Centers
Note! When checked on 1-Jan-2025, the videos at these two URLs are no longer available at their original links and were not found in the Internet Archive Wayback Machine.
For two entirely different purposes, Facebook and Google released videos recently that give glimpses into what each does to run a data center. The four-and-a-half-minute Facebook video introduces us to their Open Compute Project: a set of plans for server hardware and for physical buidings to creating the most efficient computer clusters possible. In the seven-minute Google video, we see part of what Google does to keep data safe that is stored in the cloud (including a pair of hard drive crushing machines!).