MapReduce

Dean, J., & Ghemawat, S. (2008). MapReduce: Simplified Data Processing on Large Clusters. Communications of the ACM, 51(1), 107-113.

Type

Journal article

Intellectual base

Primary research

Year

2008

DOI

10.1145/1327452.1327492

Link

https://doi.org/10.1145/1327452.1327492

Cited from

computer_science, economics_finance, logistics_supply_chain, operations_research

Cited by

5 citations across 5 artifacts.

Each citation links to the sentence it supports in the citing article.

Primes

• Batch Processing
  • Computing and data engineering — scheduled jobs and data pipelines, grouped parallel work under one instruction, and request coalescing, all amortising a per-batch setup.

    This sourceCanonical batch data-processing model: grouped parallel work amortizing per-job setup over large item sets.

• Bottleneck

  • This sourceBiblio reference (distributed-computing cluster, never cited in body).

• Concurrency

  • This sourceOSDI '04). Parallel map/shuffle/reduce across worker clusters. Tier C (bibliography only). DOI verified.

• Division of Labor
  • MapReduce (Dean and Ghemawat, 2008) is the canonical computational instance: a large data-processing task is partitioned into independent map sub-tasks, assigned to differentiated worker nodes, executed in parallel, and re-integrated through the reduce phase — the five-role pattern realized in pure computational substrate with no human performer in sight.
• Scalability

  • This sourceDescribes Google's MapReduce framework: a large data-processing task is partitioned into independent map sub-tasks executed in parallel across thousands of worker nodes and re-integrated through a shuffle-and-reduce phase. Canonical computational instance of partition-assign-execute-reintegrate division of labor in a purely silicon substrate.

Verification

This reference passed the adversarial substantiation pipeline: it was checked to exist and to support the claim it is attached to. See how references were verified.

Registry ID ref:bf03321ea85c · see in the full table