Thursday Threads: Cloud Computing and Data Centers — Amazon, Facebook, and Google

Receive DLTJ Thursday Threads:

by E-mail

by RSS

Delivered by FeedBurner

This week’s DLTJ Thursday Threads is about data centers — those dark rooms with all of the blinking lights of computers doing our bidding. Data centers hit the mainstream news this week with the outage at one of Amazon’s cloud computing clusters. And since computers and their associated peripherals consume a lot of energy, researchers are proposing to run data centers on renewable energy. And finally Facebook and Google release separate videos that give glimpses into how large data centers are run.

Feel free to send this to others you think might be interested in the topics. If you find these threads interesting and useful, you might want to add the Thursday Threads RSS Feed to your feed reader or subscribe to e-mail delivery using the form to the right. If you would like a more raw and immediate version of these types of stories, watch my FriendFeed stream (or subscribe to its feed in your feed reader). Comments and tips, as always, are welcome.

Amazon EC2 Outage Hobbles Websites

Amazon Web Services’ Elastic Compute Cloud, which offers computation as a service to thousands of businesses, and its Relational Database Service, began experiencing errors shortly before 2 a.m. PDT on Thursday at Amazon’s US-EAST data center in Virginia and the service interruption has been ongoing for more than nine hours now.

The technical problems have slowed or disabled access to the websites of customers utilizing AWS US-East resources, including Engine Yard, Foursquare, Hootsuite, Heroku, Quora, and Reddit, to name a few.

- Amazon EC2 Outage Hobbles Websites, by Thomas Claburn, InformationWeek

Failures of Amazon’s Elastic Compute Cloud service — think of it as renting virtual computer servers somewhere out there on the internet — last week caused major internet sites to shut down. As of this writing, the root cause analysis hasn’t been published, but signs are pointing to a cascade of events starting with a minor failure that snowballed into system overload as the rented servers tried to restart themselves in other areas of Amazon’s cloud capacity. The questions being raised though are leading to a darkening of the puffy white cloud computing promise. Ultimately, though, use of computing in the cloud seems to be a trade-off where you can save money by not owning your own computing infrastructure with the downside that you don’t have as much control when something goes wrong.

Far-flung Data Centers Could Use Otherwise Unharvestable Renewable Energy For Computation

Researchers at Cambridge University want to put data centers in places so remote they aren’t on any power grid. Their models indicate that moving data-hungry computation to places such as scorching deserts, windswept peaks, and the middle of the Atlantic Ocean — all rich in sunlight and wind energy — could allow this otherwise unharvestable energy to do useful work.- Really Remote Data, by Christopher Mims, MIT Technology Review

The second thread comes by way of MIT Technology Review and points to a paper by Sherif Akoush, Ripduman Sohan, Andrew Rice, Andrew W. Moore and Andy Hopper — all of Cambridge University called Free Lunch: Exploiting Renewable Energy For Computing, to be presented at the USENIX-sponsored the 13th Workshop on Hot Topics in Operating Systems next month. The “Free Lunch” part comes from using renewable energy sources at these various locations to power data centers where compute jobs are shuffled around the locations depending on the available energy — and consequently computing capacity — at each center. A neat idea, and one that is probably valuable for compute-intensive jobs like video conversion and data mining.

What Goes Into Running Large Data Centers

Facebook’s Open Compute Project

Google’s Data Center Security

Inspired by the model of open source software, we want to share the innovations in our data center for the entire industry to use and improve upon. Today we’re also announcing the formation of the Open Compute Project, an industry-wide initiative to share specifications and best practices for creating the most energy efficient and economical data centers.>

This video tour of a Google data center highlights the security and data protections that are in place at our data centers.

For two entirely different purposes, Facebook and Google released videos recently that give glimpses into what each does to run a data center. The four-and-a-half-minute Facebook video introduces us to their Open Compute Project: a set of plans for server hardware and for physical buidings to creating the most efficient computer clusters possible. In the seven-minute Google video, we see part of what Google does to keep data safe that is stored in the cloud (including a pair of hard drive crushing machines!).