WebMapReduce is a simple web-based user interface for creating and submitting Hadoop Map-Reduce jobs in practically any language. It is ideally suited for use in the introductory computer science classroom, requiring very little programming experience to write massively parallel programs.
Some of its features include:
- Simplified Map-Reduce model: WebMapReduce offers the features of Map-Reduce that are crucial to the core concept, without details that add to the learning curve.
- Extensible language support: Mappers and reducers can be written in practically any language. Python, Scheme, Java, C#, C, and C++ are all supported out of the box. All that is needed to support a new language is a simple wrapper library, possibly with an API for users to easily perform common tasks such as string processing.
- Adaptable APIs: In addition, preincluded languages can be customized to alter their APIs—for example, in order to support libraries already introduced in a class, or to suit a particular teaching style or programming paradigm. Purely functional, imperative, or object-oriented strategies are all fair game.
With WebMapReduce, writing a map-reduce job can be as simple as the following example in Python:
def mapper(key, val):
words = key.split()
for word in words:
Wmr.emit(word, '1')
def reducer(word, counts):
total = 0
for count in counts:
total += int(count)
Wmr.emit(word, count)
WebMapReduce is part of CS in Parallel, an NSF-supported project to help inject parallelism throughout the undergraduate computer science curriculum. There, you can find free teaching materials for many topics related to parallelism, including using WebMapReduce in introductory courses. For more information, see the Education page.
News
-
WebMapReduce 2.0 Available
This version moves the frontend from a PHP to a Python Django web server, includes new and improved documentation, adds support for C and C#, adds more secure user registration, and improves on various pieces of the code. To download, see the Download page. For installation instructions, see the Administration Guide.
-
WebMapReduce 1.0.2 Available
This version fixes several bugs and improves stability. A corresponding new Amazon EC2 AMI has also been released. Read the Getting Started page and then download it or set up a cluster in the cloud.
-
WebMapReduce 1.0 Available
WebMapReduce is exiting its pre-release status with the official 1.0 release! Updates over Pre-Release 2 include packaged examples, updated documentation, and support for features like numeric sorting.
We are also proud to announce the release of the CS in Parallel site, which offers resources for educators teaching parallelism—including classroom materials for use with WebMapReduce.
