Google are doing some really interesting research and the fact that they are making a lot of it public is remarkable. A recent paper Interpreting the Data:Parallel Analysis with Sawzall discusses a new data processing language, Sawzall, and shows some fascinating results …
The paper’s abstract explains: “Very large data sets often have a flat but regular structure and span multiple disks and machines. Examples include telephone call records, network logs, and web document repositories. These large data sets are not amenable to study using traditional database techniques, if only because they can be too large to fit in a single relational database. On the other hand, many of the analyses done on them can be expressed using simple, easily distributed computations: filtering, aggregation, extraction of statistics, and so on.” Sawzall is very interesting in and of itself but there’s an example in the paper that is fascinating: Using a set of search query logs the authors built a movie showing how the queries are distributed around the world through the day on August 14, 2003.
