Map Reduce Pointers

MapReduce has gained a lot of hot air. What is it, in a layman’s words?

http://pages.cs.wisc.edu/~gibson/mapReduceTutorial.html:

The MapReduce programming model is inspired by functional languages and targets data-intensive computations. The input data format is application-specific, and is specified by the user. The output is a set of pairs. The user expresses an algorithm using two functions, Map and Reduce.

The Map function is applied on the input data and produces a list of intermediate pairs. The Reduce function is applied to all intermediate pairs with the same key. It typically performs some kind of merging operation and produces zero or more output pairs. Finally, the output pairs are sorted by their key value. In the simplest form of MapReduce programs, the programmer provides just the Map function. All other functionality, including the grouping of the intermediate pairs which have the same key and the final sorting, is provided by the runtime.

Advertisements

About minghuasweblog

a long time coder
This entry was posted in All, C/C++, Reading and tagged . Bookmark the permalink.

3 Responses to Map Reduce Pointers

  1. mailgeek says:

    A visual explanation can be seen here: http://ayende.com/blog/4435/map-reduce-a-visual-explanation. Interesting topics.

  2. mailgeek says:

    Machine and human readable data representation plays a key role. Search for “c++ output json” on google, check out these links:

    http://stackoverflow.com/questions/8220130/converting-c-class-to-json
    http://www.codeproject.com/Articles/20027/JSON-Spirit-A-C-JSON-Parser-Generator-Implemented
    http://stackoverflow.com/questions/245973/whats-the-best-c-json-parser

    Namely a few good C/C++ to JSON libraries:
    jsoncpp — old and stable
    json spirit — depends on boost
    boost property_tree — new
    yajl — C only, worth a try. fast.
    jason-c — C only aos, but need you take care of memory allocation, and it has horrible error log.

  3. Pingback: JSON Data Exchange Format for Improved Software Development Productivity | minghuasweblog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s