German English

Parallel Entity Resolution with Dedoop

further information
Google Scholar
Kolb, L.; Rahm, E.
Parallel Entity Resolution with Dedoop
Datenbank-Spektrum 13 (1), 2013

Weitere Informationen:


We provide an overview of Dedoop (Deduplication with Hadoop), a new tool for parallel entity resolution (ER) on cloud infrastructures. Dedoop supports a browser-based specification of complex ER strategies and provides a large library of blocking and matching approaches. To simplify the configuration of ER strategies with several similarity metrics, training-based machine learning approaches can be employed with Dedoop. Specified ER strategies are automatically translated into MapReduce jobs for parallel execution on different Hadoop clusters. For improved performance, Dedoop supports redundancy-free multi-pass blocking as well as advanced load balancing approaches. To illustrate the usefulness of Dedoop, we present the results of a comparative evaluation of different ER strategies on a challenging real-world dataset.

Please visit our project website for further informations about Dedoop.


  • MapReduce
  • Hadoop
  • Entity Resolution
  • Blocking
  • Data Skew
  • Load Balancing


  author    = {Lars Kolb and Erhard Rahm},
  title     = {{Parallel Entity Resolution with Dedoop}},
  journal   = {Datenbank-Spektrum},
  volume    = {13},
  number    = {1},
  year      = {2013},
  pages     = {23-32},
  ee        = {},
  bibsource = {DBLP,}