Thursday Threads: Cloud Computing and Data Centers -- Amazon, Facebook, and Google
This week's DLTJ Thursday Threads is about data centers -- those dark rooms with all of the blinking lights of computers doing our bidding. Data centers hit the mainstream news this week with the outage at one of Amazon's cloud computing clusters. And since computers and their associated peripherals consume a lot of energy, researchers are proposing to run data centers on renewable energy. And finally Facebook and Google release separate videos that give glimpses into how large data centers are run.
Feel free to send this newsletter to others you think might be interested in the topics. If you are not already subscribed to DLTJ's Thursday Threads, visit the sign-up page. If you would like a more raw and immediate version of these types of stories, follow me on Mastodon where I post the bookmarks I save. Comments and tips, as always, are welcome.
Amazon EC2 Outage Hobbles Websites
Amazon Web Services' Elastic Compute Cloud, which offers computation as a service to thousands of businesses, and its Relational Database Service, began experiencing errors shortly before 2 a.m. PDT on Thursday at Amazon's US-EAST data center in Virginia and the service interruption has been ongoing for more than nine hours now.
The technical problems have slowed or disabled access to the websites of customers utilizing AWS US-East resources, including Engine Yard, Foursquare, Hootsuite, Heroku, Quora, and Reddit, to name a few.
- Amazon EC2 Outage Hobbles Websites, by Thomas Claburn, InformationWeek
Failures of Amazon's Elastic Compute Cloud service -- think of it as renting virtual computer servers somewhere out there on the internet -- last week caused major internet sites to shut down. As of this writing, the root cause analysis hasn't been published, but signs are pointing to a cascade of events starting with a minor failure that snowballed into system overload as the rented servers tried to restart themselves in other areas of Amazon's cloud capacity. The questions being raised though are leading to a darkening of the puffy white cloud computing promise. Ultimately, though, use of computing in the cloud seems to be a trade-off where you can save money by not owning your own computing infrastructure with the downside that you don't have as much control when something goes wrong.
Far-flung Data Centers Could Use Otherwise Unharvestable Renewable Energy For Computation
Researchers at Cambridge University want to put data centers in places so remote they aren't on any power grid. Their models indicate that moving data-hungry computation to places such as scorching deserts, windswept peaks, and the middle of the Atlantic Ocean — all rich in sunlight and wind energy — could allow this otherwise unharvestable energy to do useful work.- Really Remote Data, by Christopher Mims, MIT Technology Review
The second thread comes by way of MIT Technology Review and points to a paper by Sherif Akoush, Ripduman Sohan, Andrew Rice, Andrew W. Moore and Andy Hopper -- all of Cambridge University called Free Lunch: Exploiting Renewable Energy For Computing, to be presented at the USENIX-sponsored the 13th Workshop on Hot Topics in Operating Systems next month. The "Free Lunch" part comes from using renewable energy sources at these various locations to power data centers where compute jobs are shuffled around the locations depending on the available energy -- and consequently computing capacity -- at each center. A neat idea, and one that is probably valuable for compute-intensive jobs like video conversion and data mining.
What Goes Into Running Large Data Centers
Facebook's Open Compute Project
Google's Data Center Security
Inspired by the model of open source software, we want to share the innovations in our data center for the entire industry to use and improve upon. Today we’re also announcing the formation of the Open Compute Project, an industry-wide initiative to share specifications and best practices for creating the most energy efficient and economical data centers.>
This video tour of a Google data center highlights the security and data protections that are in place at our data centers.
For two entirely different purposes, Facebook and Google released videos recently that give glimpses into what each does to run a data center. The four-and-a-half-minute Facebook video introduces us to their Open Compute Project: a set of plans for server hardware and for physical buidings to creating the most efficient computer clusters possible. In the seven-minute Google video, we see part of what Google does to keep data safe that is stored in the cloud (including a pair of hard drive crushing machines!).