UW's computer sciences plays a big role in Google's data storage

When Google recently announced it would be upgrading its systems to use the Linux ext4 open-source file system, a key piece of technology developed at the UW-Madison Computer Sciences Department became a key part of the world's data-storage infrastructure.

Virtually all data within Google, including email messages, YouTube videos, and the large search database that is accessed by millions of Google search requests every day, is stored across many thousands of machines running Linux.

Each of those machines stores data in what is known as a file system which is responsible for reliably and efficiently storing and retrieving the data from hard-disk drives.

The Advanced Data Systems Lab at UW-Madison, co-led by Professors Andrea Arpaci-Dusseau and Remzi Arpaci-Dusseau, specializes in building reliable and efficient storage.

In an effort led by Vijayan Prabhakaran (MS'03, PhD'06, Computer Sciences) the group developed a new technique for writing data to disk more reliably and efficiently, known as the "transactional checksum." 

This technique ensures that every write to disk is committed safely and with little overhead — improving the ability of the file system to recover in the presence of system crashes and other typical hardware failures, while simultaneously improving performance.

As Linux ext4 has matured, it has slowly gained momentum, and the adoption by Google cements its place in the mainstream.

Next time you read your Gmail, access a Google document, or search, rest assured that beneath it all, your data is being carefully guarded by advances in storage technology developed at UW-Madison.

Category: Biological & Physical Sciences  | 
Tags: ,